Jul 23 22:20:46.553791 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 22:20:46.553804 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 22:20:46.553814 ip-10-0-133-188 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 22:20:46.554138 ip-10-0-133-188 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 22:20:56.781468 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 22:20:56.781485 ip-10-0-133-188 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e666307cb59a4c908fbd6a95e0b6f542 --
Jul 23 22:22:49.531188 ip-10-0-133-188 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 22:22:49.877765 ip-10-0-133-188 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 22:22:49.877765 ip-10-0-133-188 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 22:22:49.877765 ip-10-0-133-188 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 22:22:49.877765 ip-10-0-133-188 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 22:22:49.877765 ip-10-0-133-188 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 22:22:49.879221 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.879097    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 22:22:49.883128 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883113    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.883128 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883128    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883132    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883135    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883137    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883142    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883145    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883148    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883151    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883153    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883156    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883159    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883161    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883164    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883167    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883170    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883172    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883175    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883177    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883180    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883182    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.883213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883185    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883187    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883190    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883192    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883195    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883197    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883200    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883202    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883205    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883207    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883210    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883212    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883215    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883218    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883228    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883230    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883233    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883236    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883238    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883241    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.883725 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883243    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883246    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883248    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883250    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883253    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883255    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883258    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883260    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883262    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883265    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883268    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883270    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883272    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883275    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883279    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883281    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883284    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883288    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883293    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883296    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.884221 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883299    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883301    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883304    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883307    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883309    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883312    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883314    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883317    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883319    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883322    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883327    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883330    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883332    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883335    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883337    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883340    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883342    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883345    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883347    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883349    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.884699 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883352    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883355    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883357    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883360    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883362    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883364    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883368    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883371    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883373    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883376    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883378    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883382    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883386    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883389    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883391    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883394    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883396    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.883399    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884113    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.885213 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884119    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884122    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884125    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884128    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884131    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884134    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884137    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884140    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884142    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884145    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884148    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884151    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884154    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884156    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884159    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884161    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884165    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884168    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884171    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884174    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.885662 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884176    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884179    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884181    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884184    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884186    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884189    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884192    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884194    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884197    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884199    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884201    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884204    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884206    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884209    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884211    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884214    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884216    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884219    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884221    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884224    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.886202 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884227    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884230    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884233    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884235    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884238    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884240    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884243    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884245    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884248    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884250    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884252    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884255    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884258    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884260    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884263    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884266    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884269    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884273    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884276    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.886704 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884279    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884281    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884284    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884287    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884289    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884292    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884294    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884297    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884299    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884301    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884304    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884308    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884310    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884312    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884315    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884338    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884344    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884347    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884350    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884352    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.887184 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884355    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884357    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884361    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884365    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884368    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884371    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884373    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884376    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884379    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884382    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884385    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884390    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884392    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884395    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884397    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884400    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884402    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884404    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884407    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.887668 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884775    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884780    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884783    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884785    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884788    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884791    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884793    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884796    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884798    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884803    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884807    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884810    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884812    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884815    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884817    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884820    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884823    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884826    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884828    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.888147 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884831    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884833    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884836    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884838    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884840    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884845    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884847    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884849    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884852    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884854    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884857    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884859    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884863    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884866    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884869    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884872    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884874    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884879    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884882    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.888619 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884885    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884887    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884905    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884908    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884911    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884913    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884916    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884918    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884921    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884923    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884926    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884928    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884931    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884933    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884936    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884939    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884941    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884943    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884947    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884950    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.889109 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884952    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884955    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884957    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884960    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884962    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884965    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884967    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884971    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884973    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884976    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884988    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884992    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884995    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.884998    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885001    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885004    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885007    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885009    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885012    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885015    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.889591 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885017    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885020    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885023    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885026    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885030    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885034    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885037    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885039    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885042    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885045    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885049    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885051    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885053    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885056    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885058    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885061    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885063    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885066    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885068    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.890075 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885071    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885074    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885137    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885145    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885151    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885155    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885159    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885162    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885166    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885171    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885174    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885177    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885181    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885184    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885187    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885190    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885193    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885195    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885198    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885201    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885206    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885209    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885212    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885216    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 22:22:49.890532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885220    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885223    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885227    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885230    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885233    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885236    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885240    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885242    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885246    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885248    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885252    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885257    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885259    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885262    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885265    2559 flags.go:64] FLAG: --enable-server="true"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885268    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885273    2559 flags.go:64] FLAG: --event-burst="100"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885276    2559 flags.go:64] FLAG: --event-qps="50"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885279    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885281    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885284    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885288    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885291    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885294    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885296    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 23 22:22:49.891096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885299    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885302    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885305    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885308    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885311    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885313    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885317    2559 flags.go:64] FLAG: --feature-gates=""
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885320    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885324    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885327    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885330    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885333    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885336    2559 flags.go:64] FLAG: --help="false"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885339    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-133-188.ec2.internal"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885342    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885345    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885347    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885351    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885357    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885360    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885363    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885366    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885368    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 22:22:49.891763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885372    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885375    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885378    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885381    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885384    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885387    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885389    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885392    2559 flags.go:64] FLAG: --lock-file=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885395    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885398    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885401    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885406    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885409    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885411    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885414    2559 flags.go:64] FLAG: --logging-format="text"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885417    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885421    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885423    2559 flags.go:64] FLAG: --manifest-url=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885426    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885431    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885434    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885442    2559 flags.go:64] FLAG: --max-pods="110"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885445    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885448    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885450    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 22:22:49.892359 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885453    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885456    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885460    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885463    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885470    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885473    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885476    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885479    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885483    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885488    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885491    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885494    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885497    2559 flags.go:64] FLAG: --port="10250"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885500    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885503    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-013cd1b64a9841910"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885506    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885509    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885512    2559 flags.go:64] FLAG: --register-node="true"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885515    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885518    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885521    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885524    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885527    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885531    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885534    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 22:22:49.893025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885537    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885540    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885543    2559 flags.go:64] FLAG: --runonce="false"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885545    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885548    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885551    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885554    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885556    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885559    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885562    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885566    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885569    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885572    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885575    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885578    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885581    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885584    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885587    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885592    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885595    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885598    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885602    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885605    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885607    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885610    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 22:22:49.893634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885613    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885615    2559 flags.go:64] FLAG: --v="2"
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885619    2559 flags.go:64] FLAG: --version="false"
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885623    2559 flags.go:64] FLAG: --vmodule=""
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885627    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.885630    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885739    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885744    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885747    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885750    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885753    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885755    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885758    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885760    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885762    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885765    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885768    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885771    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885773    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885776    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885778    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.894361 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885781    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885783    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885786    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885789    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885791    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885793    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885796    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885798    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885800    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885803    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885805    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885807    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885809    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885812    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885818    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885821    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885824    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885826    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885829    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.894866 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885831    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885833    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885837    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885840    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885843    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885845    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885848    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885851    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885853    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885856    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885859    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885861    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885864    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885866    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885869    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885871    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885874    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885877    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885879    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.895384 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885882    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885884    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885886    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885903    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885906    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885908    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885911    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885913    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885917    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885919    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885922    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885924    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885927    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885930    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885932    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885935    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885937    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885940    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885942    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885945    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.895840 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885947    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885949    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885952    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885955    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885957    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885959    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885962    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885964    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885967    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885969    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885971    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885977    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885980    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885982    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885985    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885987    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885990    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885992    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885994    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.885997    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.896398 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886000    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886003    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886005    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886007    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886010    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.886012    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.886509    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893076    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893090    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893110    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893116    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893171    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893175    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893178    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893181    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.896872 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893185    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893188    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893191    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893193    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893196    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893198    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893201    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893203    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893206    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893209    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893212    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893214    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893217    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893219    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893221    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893224    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893226    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893229    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893232    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893235    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.897338 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893237    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893240    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893242    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893245    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893247    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893249    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893252    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893255    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893257    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893260    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893262    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893264    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893267    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893270    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893274    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893277    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893280    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893283    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893285    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893288    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.897928 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893290    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893293    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893296    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893299    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893301    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893304    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893306    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893308    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893311    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893313    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893316    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893318    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893321    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893323    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893326    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893328    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893330    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893333    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893336    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.898706 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893338    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893341    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893343    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893345    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893348    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893350    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893352    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893355    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893357    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893359    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893362    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893364    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893366    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893369    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893371    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893374    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893376    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893379    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893381    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893383    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.899442 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893385    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893388    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893392    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893395    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893398    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893401    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893404    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893406    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893409    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893411    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893414    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893417    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893421    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893423    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893426    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.900102 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893428    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893433    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893551    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893555    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893558    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893561    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893564    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893567    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893569    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893572    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893575    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893578    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893580    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893583    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893586    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 22:22:49.900557 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893589    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893592    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893595    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893597    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893600    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893602    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893606    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893609    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893612    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893615    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893617    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893620    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893623    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893625    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893628    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893630    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893633    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893637    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 22:22:49.901131 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893640    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893642    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893645    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893647    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893650    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893652    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893655    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893657    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893659    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893662    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893664    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893666    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893669    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893674    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893676    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893679    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893681    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893683    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893686    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893688    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 22:22:49.901648 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893690    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893693    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893695    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893697    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893700    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893702    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893707    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893709    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893711    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893713    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893716    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893718    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893721    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893723    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893725    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893728    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893731    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893734    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893737    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 22:22:49.902143 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893740    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893743    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893746    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893749    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893751    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893753    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893756    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893759    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893762    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893765    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893767    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893770    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893773    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893775    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893779    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893782    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893784    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893787    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893790    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893794    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 22:22:49.902596 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893796    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893799    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893801    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893803    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893806    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893809    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893811    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893813    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:49.893816    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893821    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.893939    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.895805    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.897266    2559 server.go:1019] "Starting client certificate rotation"
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.897388    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 22:22:49.903093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.897433    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 22:22:49.903519 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.899819    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 22:22:49.906493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.906473    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 22:22:49.916691 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.916669    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 23 22:22:49.918493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.918477    2559 log.go:25] "Validated CRI v1 image API"
Jul 23 22:22:49.919281 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.919267    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 22:22:49.921580 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.921554    2559 fs.go:135] Filesystem UUIDs: map[724f4aea-a9fa-4a34-beb3-44892b9fcef6:/dev/nvme0n1p4 78b6656d-0273-41e9-8659-28f78c540cce:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 23 22:22:49.921624 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.921580    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 22:22:49.926446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.926310    2559 manager.go:216] Machine: {Timestamp:2026-07-23 22:22:49.925143392 +0000 UTC m=+0.288809789 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104746 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec226b6c924cac658afacc91a4c5f64c SystemUUID:ec226b6c-924c-ac65-8afa-cc91a4c5f64c BootID:e666307c-b59a-4c90-8fbd-6a95e0b6f542 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:96:cb:f6:f1:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:96:cb:f6:f1:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:44:6e:24:a1:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 22:22:49.926446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.926442    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 22:22:49.926555 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.926526    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 22:22:49.929085 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.929066    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 22:22:49.929382 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.929353    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 22:22:49.929518 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.929385    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 22:22:49.930305 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.930294    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 22:22:49.930358 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.930308    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 22:22:49.930358 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.930316    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 22:22:49.930887 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.930875    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 22:22:49.932264 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.932252    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 22:22:49.934817 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934805    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 22:22:49.934941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934931    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 22:22:49.934981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934961    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 22:22:49.934981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934972    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 22:22:49.934981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934982    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 22:22:49.935086 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934990    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 23 22:22:49.935086 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.934999    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 22:22:49.935425 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.935413    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 22:22:49.935464 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.935432    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 22:22:49.937367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.937348    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 22:22:49.938867 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.938850    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 22:22:49.938971 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.938906    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 22:22:49.939854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939841    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939862    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939872    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939880    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939912    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939921    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939933    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 22:22:49.939938 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939941    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939952    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939960    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939972    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.939980    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.940013    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 22:22:49.940247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.940021    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 22:22:49.943291 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.943276    2559 server.go:1268] "Started kubelet"
Jul 23 22:22:49.946667 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.946318    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 22:22:49.946730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.946679    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 22:22:49.947201 ip-10-0-133-188 systemd[1]: Started Kubernetes Kubelet.
Jul 23 22:22:49.947465 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.943385    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 22:22:49.948550 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.948528    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 22:22:49.949350 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.949321    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 22:22:49.950248 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.950218    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 22:22:49.951182 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.951168    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 22:22:49.952413 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.952392    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 22:22:49.953977 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.953954    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 22:22:49.954163 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954152    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 22:22:49.954227 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954167    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 22:22:49.954287 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.954228    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:49.954625 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954609    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 22:22:49.954625 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954611    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 22:22:49.954747 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954637    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 22:22:49.954798 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954754    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 22:22:49.954798 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954762    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 22:22:49.954923 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.954009    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18c50b684e77cbe7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-07-23 22:22:49.943247847 +0000 UTC m=+0.306914221,LastTimestamp:2026-07-23 22:22:49.943247847 +0000 UTC m=+0.306914221,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Jul 23 22:22:49.954923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954841    2559 factory.go:55] Registering systemd factory
Jul 23 22:22:49.954923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.954855    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 23 22:22:49.955141 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955103    2559 factory.go:153] Registering CRI-O factory
Jul 23 22:22:49.955141 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955118    2559 factory.go:223] Registration of the crio container factory successfully
Jul 23 22:22:49.955202 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955167    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 22:22:49.955202 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955190    2559 factory.go:103] Registering Raw factory
Jul 23 22:22:49.955261 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955206    2559 manager.go:1196] Started watching for new ooms in manager
Jul 23 22:22:49.955485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955463    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nx5jl"
Jul 23 22:22:49.955647 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.955634    2559 manager.go:349] Starting recovery of all containers
Jul 23 22:22:49.958929 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.958742    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 22:22:49.959263 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.958926    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 22:22:49.964080 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.964054    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nx5jl"
Jul 23 22:22:49.965243 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.965218    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 22:22:49.965447 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.965432    2559 manager.go:354] Recovery completed
Jul 23 22:22:49.970684 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.970527    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:49.973092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973073    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:49.973170 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973109    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:49.973170 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973129    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:49.973664 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973651    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 22:22:49.973664 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973663    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 22:22:49.973763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973678    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 22:22:49.973763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.973683    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:49.974469 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974455    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:49.974532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974482    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:49.974532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974495    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:49.974731 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974719    2559 policy_none.go:49] "None policy: Start"
Jul 23 22:22:49.974764 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974738    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 22:22:49.974764 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.974751    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 22:22:49.975086 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:49.975021    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18c50b68503f2d49  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-07-23 22:22:49.973091657 +0000 UTC m=+0.336758037,LastTimestamp:2026-07-23 22:22:49.973091657 +0000 UTC m=+0.336758037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Jul 23 22:22:49.975803 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:49.975792    2559 policy_none.go:47] "Start"
Jul 23 22:22:50.017525 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.017506    2559 manager.go:337] "Starting Device Plugin manager"
Jul 23 22:22:50.017627 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.017544    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 22:22:50.017627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.017558    2559 server.go:87] "Starting device plugin registration server"
Jul 23 22:22:50.017827 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.017814    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 22:22:50.017868 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.017829    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 22:22:50.017953 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.017936    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 22:22:50.018017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.018010    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 22:22:50.018073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.018019    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 22:22:50.019172 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.019149    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 22:22:50.019266 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.019204    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.103678 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.103645    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 22:22:50.104663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.103694    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 22:22:50.104663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.103722    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 22:22:50.104663 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.103762    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 22:22:50.106235 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.106217    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:50.118198 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.118170    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:50.118994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.118977    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:50.119085 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.119014    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:50.119085 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.119028    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:50.119085 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.119059    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.127498 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.127481    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.127582 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.127506    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-188.ec2.internal\": node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.141167 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.141114    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.204736 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.204683    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"]
Jul 23 22:22:50.204942 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.204782    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:50.206642 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.206622    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:50.206755 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.206652    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:50.206755 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.206662    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:50.207879 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.207865    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:50.208000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.207985    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.208047 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208032    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:50.208677 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208658    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:50.208752 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208690    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:50.208752 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208707    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:50.208752 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208719    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:50.208842 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208693    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:50.208842 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.208775    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:50.209837 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.209824    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.209921 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.209849    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 22:22:50.210513 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.210500    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 22:22:50.210566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.210525    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 22:22:50.210566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.210538    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Jul 23 22:22:50.233446 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.233430    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-188.ec2.internal\" not found" node="ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.236787 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.236771    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-188.ec2.internal\" not found" node="ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.241297 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.241281    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.256261 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.256237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.256332 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.256263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.256332 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.256279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e80edc15aa7c49c99a25a6d073ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"36e80edc15aa7c49c99a25a6d073ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.341550 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.341516    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.356899 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.356874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.356984 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.356918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.356984 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.356936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e80edc15aa7c49c99a25a6d073ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"36e80edc15aa7c49c99a25a6d073ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.357054 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.356995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.357088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.357055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42d08184305b2a4a58eeaf23d57bf792-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"42d08184305b2a4a58eeaf23d57bf792\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.357119 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.357098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e80edc15aa7c49c99a25a6d073ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"36e80edc15aa7c49c99a25a6d073ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.442416 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.442339    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.535994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.535956    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.539765 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.539509    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:50.542602 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.542579    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.647146 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.647093    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.748778 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.748731    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.788886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.788858    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:50.853000 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.852953    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.889061 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.889029    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:50.896530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.896509    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 22:22:50.896694 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.896671    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 22:22:50.896742 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.896700    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 22:22:50.896789 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.896733    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 22:22:50.953572 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:50.953533    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:50.970680 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.970415    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 22:17:50 +0000 UTC" deadline="2028-03-11 20:04:44.615807474 +0000 UTC"
Jul 23 22:22:50.970680 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:50.970463    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14325h41m53.645349021s"
Jul 23 22:22:51.054424 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.054395    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:51.065574 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:51.065533    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d08184305b2a4a58eeaf23d57bf792.slice/crio-6c5d4c9e7af7f334cd273fe93bbb7129be39d07c8f5bbb89b6e85a838cc2c447 WatchSource:0}: Error finding container 6c5d4c9e7af7f334cd273fe93bbb7129be39d07c8f5bbb89b6e85a838cc2c447: Status 404 returned error can't find the container with id 6c5d4c9e7af7f334cd273fe93bbb7129be39d07c8f5bbb89b6e85a838cc2c447
Jul 23 22:22:51.072601 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.072583    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 22:22:51.097498 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:51.097471    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e80edc15aa7c49c99a25a6d073ca65.slice/crio-61b2591e7865c7764d3ebb156dcfd34e412a62f93545c50047e032bca8cf4d46 WatchSource:0}: Error finding container 61b2591e7865c7764d3ebb156dcfd34e412a62f93545c50047e032bca8cf4d46: Status 404 returned error can't find the container with id 61b2591e7865c7764d3ebb156dcfd34e412a62f93545c50047e032bca8cf4d46
Jul 23 22:22:51.107402 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.107355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" event={"ID":"36e80edc15aa7c49c99a25a6d073ca65","Type":"ContainerStarted","Data":"61b2591e7865c7764d3ebb156dcfd34e412a62f93545c50047e032bca8cf4d46"}
Jul 23 22:22:51.108440 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.108415    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"42d08184305b2a4a58eeaf23d57bf792","Type":"ContainerStarted","Data":"6c5d4c9e7af7f334cd273fe93bbb7129be39d07c8f5bbb89b6e85a838cc2c447"}
Jul 23 22:22:51.157080 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.157034    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:51.261384 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.261343    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Jul 23 22:22:51.268563 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.268543    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:51.355464 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.355382    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:51.368025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.367999    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 22:22:51.369500 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.369484    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Jul 23 22:22:51.381227 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.381205    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 22:22:51.938383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.938341    2559 apiserver.go:52] "Watching apiserver"
Jul 23 22:22:51.944982 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.944957    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 22:22:51.946858 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.946829    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-5xft7","kube-system/konnectivity-agent-74qqm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc","openshift-image-registry/node-ca-mdgfz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal","openshift-multus/multus-additional-cni-plugins-8pl4b","openshift-multus/network-metrics-daemon-pzvld","openshift-network-operator/iptables-alerter-bvw98","openshift-ovn-kubernetes/ovnkube-node-gnptj","kube-system/global-pull-secret-syncer-x5h2b","kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7zrkj","openshift-dns/node-resolver-z4t9t","openshift-multus/multus-5qjfk"]
Jul 23 22:22:51.948477 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.948455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:51.948579 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.948541    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:22:51.950080 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.950055    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:51.951277 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.951250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.953044 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.953023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:51.954149 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.954124    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:51.954247 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.954194    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:22:51.954994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.954971    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 22:22:51.955553 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.955534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956372    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956511    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d6tlr\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956561    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956689    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956820    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.956852    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957054    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9j4g7\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957188    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957213    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957358    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-49drd\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957420    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.957645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.957542    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.958228 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958012    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:51.958608 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958570    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 22:22:51.958728 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958623    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 22:22:51.958728 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958637    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.958879 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958862    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.958919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.958865    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xdkcq\""
Jul 23 22:22:51.959086 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.959068    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 22:22:51.960391 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.959427    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:51.961254 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.961236    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:51.963010 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.962962    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 22:22:51.963222 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.963205    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 22:22:51.963312 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.963291    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.963489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.963473    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 22:22:51.963647 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.963634    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.963846 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.963828    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.964147 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.964129    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.964292 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.964274    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 22:22:51.964626 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.964611    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tkkw9\""
Jul 23 22:22:51.964994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.964969    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pqsdz\""
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.965996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:51.966074    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7cb5fd5-971d-490e-b70e-7c2c1397360a-host-slash\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-socket-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bec428-512b-40ae-b77a-eb0dbbd99810-serviceca\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-system-cni-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cnibin\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx8n6\" (UniqueName: \"kubernetes.io/projected/e07a8b0e-1d52-4822-8d02-4a7f82b83486-kube-api-access-rx8n6\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966481    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjnp\" (UniqueName: \"kubernetes.io/projected/b7cb5fd5-971d-490e-b70e-7c2c1397360a-kube-api-access-lgjnp\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-device-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bec428-512b-40ae-b77a-eb0dbbd99810-host\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7gp\" (UniqueName: \"kubernetes.io/projected/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-kube-api-access-mz7gp\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966599    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.966812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966630    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nsnn\" (UniqueName: \"kubernetes.io/projected/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kube-api-access-9nsnn\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nz64\" (UniqueName: \"kubernetes.io/projected/36bec428-512b-40ae-b77a-eb0dbbd99810-kube-api-access-4nz64\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.966818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.967689 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7cb5fd5-971d-490e-b70e-7c2c1397360a-iptables-alerter-script\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:51.968053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-registration-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.968053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967654    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:51.968053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-sys-fs\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:51.968053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.967827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-os-release\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:51.968479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.968456    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 22:22:51.968591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.968510    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sbtvf\""
Jul 23 22:22:51.968673 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.968654    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 22:22:51.970839 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.970818    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:51.971278 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.971256    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 22:22:51.971360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.971317    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g4xrb\""
Jul 23 22:22:51.973877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.973135    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 22:22:51.973877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.973316    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t47xv\""
Jul 23 22:22:51.973877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.973355    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 22:22:51.978417 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:51.978393    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 22:22:52.006673 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.006647    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7s5hg"
Jul 23 22:22:52.016957 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.016776    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7s5hg"
Jul 23 22:22:52.056089 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.056066    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 22:22:52.068596 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-os-release\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068714 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7cb5fd5-971d-490e-b70e-7c2c1397360a-host-slash\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.068714 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-socket-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.068714 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068654    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-netns\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.068714 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-log-socket\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068710    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7cb5fd5-971d-490e-b70e-7c2c1397360a-host-slash\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068716    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-netd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068739    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-os-release\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovn-node-metrics-cert\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-system-cni-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-system-cni-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-socket-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068860    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cnibin\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-konnectivity-ca\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cnibin\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-dbus\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.068966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-hostroot\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.068982    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-modprobe-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-kubernetes\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7gp\" (UniqueName: \"kubernetes.io/projected/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-kube-api-access-mz7gp\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-var-lib-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-ovn\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-netns\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-multus\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysconfig\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-lib-modules\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7cb5fd5-971d-490e-b70e-7c2c1397360a-iptables-alerter-script\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-systemd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.069507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-script-lib\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07a8b0e-1d52-4822-8d02-4a7f82b83486-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-conf-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-var-lib-kubelet\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-os-release\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-socket-dir-parent\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-systemd\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t78mc\" (UniqueName: \"kubernetes.io/projected/c60ce515-3a59-4ba8-a8d4-108338862db2-kube-api-access-t78mc\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx8n6\" (UniqueName: \"kubernetes.io/projected/e07a8b0e-1d52-4822-8d02-4a7f82b83486-kube-api-access-rx8n6\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bec428-512b-40ae-b77a-eb0dbbd99810-host\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-slash\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-system-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.069982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bec428-512b-40ae-b77a-eb0dbbd99810-host\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.070347 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070037    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-k8s-cni-cncf-io\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7cb5fd5-971d-490e-b70e-7c2c1397360a-iptables-alerter-script\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nz64\" (UniqueName: \"kubernetes.io/projected/36bec428-512b-40ae-b77a-eb0dbbd99810-kube-api-access-4nz64\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-hosts-file\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e07a8b0e-1d52-4822-8d02-4a7f82b83486-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-kubelet\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-daemon-config\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-host\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-registration-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-bin\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bec428-512b-40ae-b77a-eb0dbbd99810-serviceca\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070293    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-registration-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq2p7\" (UniqueName: \"kubernetes.io/projected/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-kube-api-access-hq2p7\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-systemd-units\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-node-log\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-run\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.071111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-bin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nsnn\" (UniqueName: \"kubernetes.io/projected/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kube-api-access-9nsnn\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-config\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-sys\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070595    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bec428-512b-40ae-b77a-eb0dbbd99810-serviceca\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-kubelet\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.070736    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.070866    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:22:52.570821284 +0000 UTC m=+2.934487675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070886    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-env-overrides\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070933    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-kubelet-config\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.071740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cni-binary-copy\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.070977    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-tuned\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjnp\" (UniqueName: \"kubernetes.io/projected/b7cb5fd5-971d-490e-b70e-7c2c1397360a-kube-api-access-lgjnp\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-device-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071160    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-etc-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5fb\" (UniqueName: \"kubernetes.io/projected/800bf694-6f91-4ca1-b1b0-f6db307d684f-kube-api-access-sl5fb\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-etc-kubernetes\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-tmp-dir\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-agent-certs\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cnibin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-device-dir\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2bl9\" (UniqueName: \"kubernetes.io/projected/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-kube-api-access-m2bl9\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-conf\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071433    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-sys-fs\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-multus-certs\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.072378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071498    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.072993 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-tmp\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.072993 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.071620    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-sys-fs\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.075650 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.075616    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 22:22:52.079368 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.079348    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nz64\" (UniqueName: \"kubernetes.io/projected/36bec428-512b-40ae-b77a-eb0dbbd99810-kube-api-access-4nz64\") pod \"node-ca-mdgfz\" (UID: \"36bec428-512b-40ae-b77a-eb0dbbd99810\") " pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.079473 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.079415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7gp\" (UniqueName: \"kubernetes.io/projected/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-kube-api-access-mz7gp\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:52.079787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.079765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx8n6\" (UniqueName: \"kubernetes.io/projected/e07a8b0e-1d52-4822-8d02-4a7f82b83486-kube-api-access-rx8n6\") pod \"multus-additional-cni-plugins-8pl4b\" (UID: \"e07a8b0e-1d52-4822-8d02-4a7f82b83486\") " pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.083146 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.083125    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:22:52.083244 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.083155    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:22:52.083244 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.083170    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:52.083419 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.083250    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:22:52.583218687 +0000 UTC m=+2.946885068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:52.085542 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.085520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjnp\" (UniqueName: \"kubernetes.io/projected/b7cb5fd5-971d-490e-b70e-7c2c1397360a-kube-api-access-lgjnp\") pod \"iptables-alerter-bvw98\" (UID: \"b7cb5fd5-971d-490e-b70e-7c2c1397360a\") " pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.086011 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.085968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nsnn\" (UniqueName: \"kubernetes.io/projected/40bd30a2-fbfe-4219-bacb-ba8990f7eaaa-kube-api-access-9nsnn\") pod \"aws-ebs-csi-driver-node-n92jc\" (UID: \"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.172319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-sys\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-kubelet\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-env-overrides\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-sys\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-kubelet-config\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-kubelet\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cni-binary-copy\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172511    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-kubelet-config\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.172523 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-tuned\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172556    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-etc-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5fb\" (UniqueName: \"kubernetes.io/projected/800bf694-6f91-4ca1-b1b0-f6db307d684f-kube-api-access-sl5fb\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-etc-kubernetes\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-tmp-dir\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-agent-certs\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cnibin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2bl9\" (UniqueName: \"kubernetes.io/projected/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-kube-api-access-m2bl9\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-conf\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-multus-certs\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172786    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-tmp\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-netns\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-log-socket\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172868    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-netd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovn-node-metrics-cert\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-konnectivity-ca\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.172965 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-dbus\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.172984    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-hostroot\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-modprobe-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-kubernetes\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-var-lib-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-log-socket\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-ovn\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-etc-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-netns\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-multus\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-modprobe-d\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysconfig\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-lib-modules\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173246    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-hostroot\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-systemd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-netd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysctl-conf\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.173810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-script-lib\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cnibin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-conf-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-var-lib-kubelet\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-kubernetes\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-var-lib-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-env-overrides\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-openvswitch\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-multus\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-netns\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173415    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-os-release\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173426    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-multus-certs\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-socket-dir-parent\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-os-release\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-ovn\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-systemd\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-netns\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t78mc\" (UniqueName: \"kubernetes.io/projected/c60ce515-3a59-4ba8-a8d4-108338862db2-kube-api-access-t78mc\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.174658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-sysconfig\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-slash\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-system-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8dbe3df2-31d4-4acf-abb7-04f4fa745718-dbus\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-k8s-cni-cncf-io\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-hosts-file\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-lib-modules\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173704    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-kubelet\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-daemon-config\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-hosts-file\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-slash\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.173849    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-host\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-bin\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.173917    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:22:52.673898662 +0000 UTC m=+3.037565041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-cni-bin\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.175546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq2p7\" (UniqueName: \"kubernetes.io/projected/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-kube-api-access-hq2p7\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-systemd-units\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-var-lib-kubelet\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-node-log\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-run\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-socket-dir-parent\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-bin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-systemd\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-host\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174161    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-config\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-systemd-units\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-run-systemd\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174238    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-run-k8s-cni-cncf-io\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-system-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-daemon-config\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-conf-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.176344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-konnectivity-ca\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174455    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-script-lib\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.173127    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-cni-binary-copy\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-kubelet\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-host-var-lib-cni-bin\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-multus-cni-dir\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174560    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/800bf694-6f91-4ca1-b1b0-f6db307d684f-node-log\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c60ce515-3a59-4ba8-a8d4-108338862db2-run\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-etc-kubernetes\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.174819    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-tmp-dir\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.177390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.175299    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovnkube-config\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.178597 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.178512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/800bf694-6f91-4ca1-b1b0-f6db307d684f-ovn-node-metrics-cert\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.178990 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.178964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-etc-tuned\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.179181 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.179137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39-agent-certs\") pod \"konnectivity-agent-74qqm\" (UID: \"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39\") " pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.181028 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.180993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c60ce515-3a59-4ba8-a8d4-108338862db2-tmp\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.183023 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.182997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2bl9\" (UniqueName: \"kubernetes.io/projected/aaedd58c-f508-49aa-96a0-cc1308fcd8c5-kube-api-access-m2bl9\") pod \"multus-5qjfk\" (UID: \"aaedd58c-f508-49aa-96a0-cc1308fcd8c5\") " pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.183209 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.183186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5fb\" (UniqueName: \"kubernetes.io/projected/800bf694-6f91-4ca1-b1b0-f6db307d684f-kube-api-access-sl5fb\") pod \"ovnkube-node-gnptj\" (UID: \"800bf694-6f91-4ca1-b1b0-f6db307d684f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.183298 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.183255    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t78mc\" (UniqueName: \"kubernetes.io/projected/c60ce515-3a59-4ba8-a8d4-108338862db2-kube-api-access-t78mc\") pod \"tuned-7zrkj\" (UID: \"c60ce515-3a59-4ba8-a8d4-108338862db2\") " pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.183791 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.183769    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq2p7\" (UniqueName: \"kubernetes.io/projected/d55a6530-2c75-405e-b5d8-9ccc3b5660b7-kube-api-access-hq2p7\") pod \"node-resolver-z4t9t\" (UID: \"d55a6530-2c75-405e-b5d8-9ccc3b5660b7\") " pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.276281 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.276229    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvw98"
Jul 23 22:22:52.283247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.283218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc"
Jul 23 22:22:52.293888 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.293844    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdgfz"
Jul 23 22:22:52.299984 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.299956    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8pl4b"
Jul 23 22:22:52.308818 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.308794    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:22:52.317583 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.317561    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4t9t"
Jul 23 22:22:52.324065 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.324045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:22:52.330556 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.330535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5qjfk"
Jul 23 22:22:52.336034 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.336013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7zrkj"
Jul 23 22:22:52.366699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.366669    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:52.578116 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.577938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:52.578116 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.578093    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:52.578323 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.578179    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:22:53.57815511 +0000 UTC m=+3.941821476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:52.678571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.678528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:52.678571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:52.678573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678692    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678713    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678735    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678744    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678763    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:22:53.678743826 +0000 UTC m=+4.042410188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:52.678815 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:52.678794    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:22:53.678777969 +0000 UTC m=+4.042444350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:53.018419 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.018383    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 22:17:52 +0000 UTC" deadline="2028-01-05 18:16:07.500481913 +0000 UTC"
Jul 23 22:22:53.018419 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.018414    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12739h53m14.482073453s"
Jul 23 22:22:53.042228 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:53.042197    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bd30a2_fbfe_4219_bacb_ba8990f7eaaa.slice/crio-2d74efa9c352b60d17a6797522ad398e1a8831104b6f7ff133d31bf794b4fd75 WatchSource:0}: Error finding container 2d74efa9c352b60d17a6797522ad398e1a8831104b6f7ff133d31bf794b4fd75: Status 404 returned error can't find the container with id 2d74efa9c352b60d17a6797522ad398e1a8831104b6f7ff133d31bf794b4fd75
Jul 23 22:22:53.043089 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:53.043051    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800bf694_6f91_4ca1_b1b0_f6db307d684f.slice/crio-01ada1afba578bf3b4fab88f19de75839b10f3a88eb3c8755aa4032a1c913eeb WatchSource:0}: Error finding container 01ada1afba578bf3b4fab88f19de75839b10f3a88eb3c8755aa4032a1c913eeb: Status 404 returned error can't find the container with id 01ada1afba578bf3b4fab88f19de75839b10f3a88eb3c8755aa4032a1c913eeb
Jul 23 22:22:53.043969 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:53.043912    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07a8b0e_1d52_4822_8d02_4a7f82b83486.slice/crio-c2de6916c17f391da4d6065abfa72ff8e8db905c610913f67fd8917b3202ac24 WatchSource:0}: Error finding container c2de6916c17f391da4d6065abfa72ff8e8db905c610913f67fd8917b3202ac24: Status 404 returned error can't find the container with id c2de6916c17f391da4d6065abfa72ff8e8db905c610913f67fd8917b3202ac24
Jul 23 22:22:53.047271 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:22:53.047249    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd55a6530_2c75_405e_b5d8_9ccc3b5660b7.slice/crio-6e57b8a0378f4567eeb0323cb489755c5674097e6b7f8c3f9f415821e135d46c WatchSource:0}: Error finding container 6e57b8a0378f4567eeb0323cb489755c5674097e6b7f8c3f9f415821e135d46c: Status 404 returned error can't find the container with id 6e57b8a0378f4567eeb0323cb489755c5674097e6b7f8c3f9f415821e135d46c
Jul 23 22:22:53.102579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.102537    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:53.115356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.115325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7zrkj" event={"ID":"c60ce515-3a59-4ba8-a8d4-108338862db2","Type":"ContainerStarted","Data":"4db8a2b1198736d50a4b2ee4434efc43e0295232eff8c855fdab2a9d7626a31d"}
Jul 23 22:22:53.116596 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.116559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdgfz" event={"ID":"36bec428-512b-40ae-b77a-eb0dbbd99810","Type":"ContainerStarted","Data":"98bc5019909b61857e424a8c11cb02ccfcafae212d00cc6d814cfbd22303d2bf"}
Jul 23 22:22:53.117768 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.117736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4t9t" event={"ID":"d55a6530-2c75-405e-b5d8-9ccc3b5660b7","Type":"ContainerStarted","Data":"6e57b8a0378f4567eeb0323cb489755c5674097e6b7f8c3f9f415821e135d46c"}
Jul 23 22:22:53.118803 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.118769    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvw98" event={"ID":"b7cb5fd5-971d-490e-b70e-7c2c1397360a","Type":"ContainerStarted","Data":"2ce15e0b3da001d9ba4ce940d61168c4a27321a51d79452166e515495a4b0126"}
Jul 23 22:22:53.120038 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.119993    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5qjfk" event={"ID":"aaedd58c-f508-49aa-96a0-cc1308fcd8c5","Type":"ContainerStarted","Data":"493156cccd651faa1f62c5a0786d9661de7a6bebddac01013e29d42831f6f15f"}
Jul 23 22:22:53.121248 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.121201    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-74qqm" event={"ID":"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39","Type":"ContainerStarted","Data":"99c1963e6fd1fb362927bb4317b0bc13082be0fb31cc264968385f52a9e6298f"}
Jul 23 22:22:53.122470 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.122442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerStarted","Data":"c2de6916c17f391da4d6065abfa72ff8e8db905c610913f67fd8917b3202ac24"}
Jul 23 22:22:53.123589 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.123562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"01ada1afba578bf3b4fab88f19de75839b10f3a88eb3c8755aa4032a1c913eeb"}
Jul 23 22:22:53.124724 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.124701    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc" event={"ID":"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa","Type":"ContainerStarted","Data":"2d74efa9c352b60d17a6797522ad398e1a8831104b6f7ff133d31bf794b4fd75"}
Jul 23 22:22:53.194420 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.194386    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 22:22:53.586647 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.586605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:53.586909 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.586823    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:53.587011 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.587000    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:22:55.586975518 +0000 UTC m=+5.950641886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:53.688048 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.687988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:53.688240 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:53.688068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:53.688240 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688210    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:22:53.688240 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688238    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:22:53.688406 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688252    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:53.688406 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688259    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:53.688406 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688321    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:22:55.688300085 +0000 UTC m=+6.051966467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:53.688406 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:53.688345    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:22:55.688333886 +0000 UTC m=+6.052000259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:54.019017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.018944    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 22:17:52 +0000 UTC" deadline="2028-04-07 11:33:38.440806632 +0000 UTC"
Jul 23 22:22:54.019017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.019013    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14965h10m44.421798303s"
Jul 23 22:22:54.106653 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.106617    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:54.106835 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:54.106762    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:22:54.107237 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.107211    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:54.107352 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:54.107329    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:22:54.107424 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.107407    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:54.107500 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:54.107481    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:22:54.161917 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.161880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" event={"ID":"36e80edc15aa7c49c99a25a6d073ca65","Type":"ContainerStarted","Data":"64d881cc14ac85c39f065ef3fac2305f9bc28cf8eb05d624345b096a876226b7"}
Jul 23 22:22:54.176263 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.175633    2559 generic.go:356] "Generic (PLEG): container finished" podID="42d08184305b2a4a58eeaf23d57bf792" containerID="fb3657952ad9712423c6c3bce9eb1dc53a192f96c8bf15580c303463ff1dc1f2" exitCode=0
Jul 23 22:22:54.176263 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.175683    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"42d08184305b2a4a58eeaf23d57bf792","Type":"ContainerDied","Data":"fb3657952ad9712423c6c3bce9eb1dc53a192f96c8bf15580c303463ff1dc1f2"}
Jul 23 22:22:54.183887 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:54.183725    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" podStartSLOduration=3.183705324 podStartE2EDuration="3.183705324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:22:54.183187242 +0000 UTC m=+4.546853655" watchObservedRunningTime="2026-07-23 22:22:54.183705324 +0000 UTC m=+4.547371713"
Jul 23 22:22:55.203843 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:55.200147    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"42d08184305b2a4a58eeaf23d57bf792","Type":"ContainerStarted","Data":"c087da12257f3ebcac38a58fca8c48f4afb5392c68e8386cff6d3df633b1844b"}
Jul 23 22:22:55.609019 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:55.608962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:55.609245 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.609201    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:55.609310 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.609268    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:22:59.609249086 +0000 UTC m=+9.972915466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:55.709430 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:55.709377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:55.709608 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:55.709440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:55.709689 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.709621    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:55.709689 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.709687    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:22:59.709668729 +0000 UTC m=+10.073335095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:55.710072 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.710050    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:22:55.710072 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.710080    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:22:55.710226 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.710095    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:55.710226 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:55.710146    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:22:59.710129257 +0000 UTC m=+10.073795641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:56.106138 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:56.106105    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:56.106328 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:56.106222    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:22:56.106639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:56.106615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:56.106754 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:56.106728    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:22:56.108345 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:56.108317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:56.108470 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:56.108416    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:22:58.104064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:58.104009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:58.104064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:58.104034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:58.104583 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:58.104193    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:58.104583 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:58.104210    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:22:58.104583 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:58.104345    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:22:58.104583 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:58.104431    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:22:59.646697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:59.646653    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:22:59.647184 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.646858    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:59.647184 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.646944    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:23:07.646923783 +0000 UTC m=+18.010590161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:22:59.747360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:59.747310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:22:59.747360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:22:59.747361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:22:59.747598 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747534    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:59.747651 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747600    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:23:07.747581213 +0000 UTC m=+18.111247651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:22:59.747751 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747691    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:22:59.747751 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747707    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:22:59.747751 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747720    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:22:59.747946 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:22:59.747761    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:23:07.747747733 +0000 UTC m=+18.111414131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:00.105932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:00.105980    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:00.106070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:00.106137    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:00.106450    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:00.106505 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:00.106463    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:02.104395 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:02.104305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:02.104395 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:02.104330    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:02.104395 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:02.104337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:02.104828 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:02.104438    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:02.104828 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:02.104539    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:02.104828 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:02.104623    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:04.104187    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:04.104219    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:04.104244    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:04.104351    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:04.104387    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:04.104686 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:04.104483    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:06.104366 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:06.104335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:06.104840 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:06.104335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:06.104840 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:06.104459    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:06.104840 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:06.104551    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:06.104840 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:06.104335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:06.108175 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:06.108144    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:07.710598 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:07.710566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:07.711122 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.710702    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:23:07.711122 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.710771    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:23:23.710749609 +0000 UTC m=+34.074416011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:23:07.811252 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:07.811213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:07.811261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811348    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811356    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811373    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811385    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811404    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:23:23.811385349 +0000 UTC m=+34.175051711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:23:07.811442 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:07.811420    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:23:23.811410038 +0000 UTC m=+34.175076419 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:23:08.104285 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:08.104247    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:08.104285 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:08.104271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:08.104497 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:08.104356    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:08.104497 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:08.104394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:08.104627 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:08.104507    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:08.104627 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:08.104543    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:10.105320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:10.105279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:10.105814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:10.105400    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:10.105814 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:10.105398    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:10.105814 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:10.105485    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:10.105814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:10.105511    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:10.105814 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:10.105622    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:12.104708 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:12.104663    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:12.105360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:12.104663    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:12.105360 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:12.104806    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:12.105360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:12.104668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:12.105360 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:12.104961    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:12.105360 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:12.105044    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:13.239785 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.239420    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-74qqm" event={"ID":"9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39","Type":"ContainerStarted","Data":"c3ae13d065d31a088a4fc610975dc5edca4abe53fcbbd9c7bf534e491c2de9db"}
Jul 23 22:23:13.241035 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.241009    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="ffc5d666a1261301c5d07a8d55d8af5a0092bb4ff022f54e78cb4832419de84e" exitCode=0
Jul 23 22:23:13.241113 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.241093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"ffc5d666a1261301c5d07a8d55d8af5a0092bb4ff022f54e78cb4832419de84e"}
Jul 23 22:23:13.244328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244167    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"cc74dd90d003c1b9eadb7cf79728d96e4d67456202fa08d1d7e53d43db8649a1"}
Jul 23 22:23:13.244328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"c6baca6041dd79378dd4ed8cf1c6a05cf87a537429073dd28d9c00d1b15cafd2"}
Jul 23 22:23:13.244328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244292    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"0389f98ecf5ce5596c2980e986eeb37f329c7984031083c88dc985a2a97ed5bf"}
Jul 23 22:23:13.244513 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"5cd42148de8b4473a3fce02948de594fa7629db48acf587b969a5f65206b523b"}
Jul 23 22:23:13.244513 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244380    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"8f8f0ce07b94fd2d2fc470f123f20b712456d7cbbdc2e11e29f0a37592046d43"}
Jul 23 22:23:13.244513 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.244423    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"208a638f35c6f72720d88be8e2e96bd75f538a95a2454a5a85e10336533c0f0f"}
Jul 23 22:23:13.247312 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.247288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc" event={"ID":"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa","Type":"ContainerStarted","Data":"8194fd03c3851a931f6373256fa85502c4dc3da7ce0ddeda6171fa8612496bc9"}
Jul 23 22:23:13.248660 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.248641    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7zrkj" event={"ID":"c60ce515-3a59-4ba8-a8d4-108338862db2","Type":"ContainerStarted","Data":"d0b5a19ec5bbf8df0fb1d7df1eec10da0fcb4df4172d192391f6f7975064d313"}
Jul 23 22:23:13.249942 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.249923    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdgfz" event={"ID":"36bec428-512b-40ae-b77a-eb0dbbd99810","Type":"ContainerStarted","Data":"17511be75ca32ef1ec4c2ebeb9fb5cf0b2f55caa9e483d7f609b19006533302d"}
Jul 23 22:23:13.251152 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.251134    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4t9t" event={"ID":"d55a6530-2c75-405e-b5d8-9ccc3b5660b7","Type":"ContainerStarted","Data":"9f8b27cb2c660f67d95915920dbb9277be67991fc58ddd492d53826c002f61bc"}
Jul 23 22:23:13.252477 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.252458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5qjfk" event={"ID":"aaedd58c-f508-49aa-96a0-cc1308fcd8c5","Type":"ContainerStarted","Data":"7a288d2c2bd0a435a3403bec6763d4082dbc1b92d14d2cf3ec0a4f5bff5b690f"}
Jul 23 22:23:13.268723 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.268688    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" podStartSLOduration=22.268676913 podStartE2EDuration="22.268676913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:22:55.219432702 +0000 UTC m=+5.583099091" watchObservedRunningTime="2026-07-23 22:23:13.268676913 +0000 UTC m=+23.632343296"
Jul 23 22:23:13.299576 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.299539    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7zrkj" podStartSLOduration=3.911498762 podStartE2EDuration="23.299525052s" totalImagesPullingTime="19.38802629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:13.299412304 +0000 UTC m=+23.663078687" watchObservedRunningTime="2026-07-23 22:23:13.299525052 +0000 UTC m=+23.663191497"
Jul 23 22:23:13.299766 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.299748    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-74qqm" podStartSLOduration=11.93248638 podStartE2EDuration="23.299739539s" totalImagesPullingTime="11.367253159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:13.26927321 +0000 UTC m=+23.632939592" watchObservedRunningTime="2026-07-23 22:23:13.299739539 +0000 UTC m=+23.663405921"
Jul 23 22:23:13.373173 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.373125    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mdgfz" podStartSLOduration=3.975647394 podStartE2EDuration="23.373111118s" totalImagesPullingTime="19.397463724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:13.32892878 +0000 UTC m=+23.692595157" watchObservedRunningTime="2026-07-23 22:23:13.373111118 +0000 UTC m=+23.736777501"
Jul 23 22:23:13.435784 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.435733    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5qjfk" podStartSLOduration=3.989139585 podStartE2EDuration="23.435717394s" totalImagesPullingTime="19.446577809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:13.435703703 +0000 UTC m=+23.799370086" watchObservedRunningTime="2026-07-23 22:23:13.435717394 +0000 UTC m=+23.799383770"
Jul 23 22:23:13.436363 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.436336    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z4t9t" podStartSLOduration=4.03585604 podStartE2EDuration="23.436328654s" totalImagesPullingTime="19.400472614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:13.40395025 +0000 UTC m=+23.767616741" watchObservedRunningTime="2026-07-23 22:23:13.436328654 +0000 UTC m=+23.799995037"
Jul 23 22:23:13.699333 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.699305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-74qqm_9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:13.770725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:13.770693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-74qqm_9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:14.104274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.104193    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:14.104274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.104240    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:14.104694 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.104351    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:14.104694 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:14.104347    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:14.104694 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:14.104425    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:14.104694 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:14.104497    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:14.235021 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.234977    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:23:14.235653 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.235632    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:23:14.256909 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.256856    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvw98" event={"ID":"b7cb5fd5-971d-490e-b70e-7c2c1397360a","Type":"ContainerStarted","Data":"a15280321885ea26f260dc97be92d6763f4330b64c1e0bb4a57852e056adf016"}
Jul 23 22:23:14.257389 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.257369    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:23:14.257855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.257835    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-74qqm"
Jul 23 22:23:14.304808 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.304750    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bvw98" podStartSLOduration=4.919119612 podStartE2EDuration="24.304732332s" totalImagesPullingTime="19.38561272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:14.274190767 +0000 UTC m=+24.637857150" watchObservedRunningTime="2026-07-23 22:23:14.304732332 +0000 UTC m=+24.668398715"
Jul 23 22:23:14.505345 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:14.505312    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 22:23:15.031842 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:15.031724    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T22:23:14.505331403Z","UUID":"4c7722ad-fee3-4074-aba7-061193650a86","Handler":null,"Name":"","Endpoint":""}
Jul 23 22:23:15.033487 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:15.033461    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 22:23:15.033611 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:15.033506    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 22:23:15.262051 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:15.262011    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc" event={"ID":"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa","Type":"ContainerStarted","Data":"3a30925d961497796f1c71034402fe8121f82e3d424ee7589175d9b8f9691b22"}
Jul 23 22:23:16.104542 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.104505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:16.104721 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.104505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:16.104721 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:16.104630    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:16.104855 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:16.104737    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:16.104855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.104505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:16.104969 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:16.104858    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:16.268051 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.267961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"89e0f4bb5481c7bd24b455ecb61feb12c6b8bce3ad4c51cbf6160ee2529513ab"}
Jul 23 22:23:16.270332 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.270305    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc" event={"ID":"40bd30a2-fbfe-4219-bacb-ba8990f7eaaa","Type":"ContainerStarted","Data":"a45c133e59e2f47efaa4b39fbc8024ba49ed09adab70a934290774d95e44174f"}
Jul 23 22:23:16.292437 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:16.292174    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n92jc" podStartSLOduration=3.996885525 podStartE2EDuration="26.292154711s" totalImagesPullingTime="22.295269186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:16.291975425 +0000 UTC m=+26.655641823" watchObservedRunningTime="2026-07-23 22:23:16.292154711 +0000 UTC m=+26.655821098"
Jul 23 22:23:17.447344 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:17.447318    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4t9t_d55a6530-2c75-405e-b5d8-9ccc3b5660b7/dns-node-resolver/0.log"
Jul 23 22:23:18.104291 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.104109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:18.104428 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.104110    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:18.104428 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:18.104365    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:18.104489 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:18.104458    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:18.104489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.104109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:18.104579 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:18.104534    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:18.277404 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.277362    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="8efffa6dabdc67be5e0559cc88163078af20c51578c1cf08cf412d6fa08f1a86" exitCode=0
Jul 23 22:23:18.277551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.277442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"8efffa6dabdc67be5e0559cc88163078af20c51578c1cf08cf412d6fa08f1a86"}
Jul 23 22:23:18.281083 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.281058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" event={"ID":"800bf694-6f91-4ca1-b1b0-f6db307d684f","Type":"ContainerStarted","Data":"582eba556b660d3621e1ea1727251883758f16ad01a04e56eb8589b78717fe61"}
Jul 23 22:23:18.281299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.281275    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:18.281402 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.281360    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:18.281452 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.281408    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:18.297414 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.297391    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:18.297535 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.297500    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:18.330096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.330053    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj" podStartSLOduration=8.855367389 podStartE2EDuration="28.330039191s" totalImagesPullingTime="19.474671802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:18.329337362 +0000 UTC m=+28.693003777" watchObservedRunningTime="2026-07-23 22:23:18.330039191 +0000 UTC m=+28.693705573"
Jul 23 22:23:18.442207 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:18.442131    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdgfz_36bec428-512b-40ae-b77a-eb0dbbd99810/node-ca/0.log"
Jul 23 22:23:19.694416 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.694331    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5xft7"]
Jul 23 22:23:19.695071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.694476    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:19.695071 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:19.694555    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:19.696960 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.696927    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5h2b"]
Jul 23 22:23:19.697096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.697034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:19.697177 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:19.697153    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:19.697681 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.697654    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pzvld"]
Jul 23 22:23:19.697781 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:19.697772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:19.697914 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:19.697873    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:20.287739 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:20.287708    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="e260e9e154164930c02ed32a5fafad0b9c1a8fdb872245dd0f14e37141287142" exitCode=0
Jul 23 22:23:20.287950 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:20.287801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"e260e9e154164930c02ed32a5fafad0b9c1a8fdb872245dd0f14e37141287142"}
Jul 23 22:23:21.104714 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:21.104681    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:21.105365 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:21.104787    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:22.104807 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:22.104623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:22.105038 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:22.104687    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:22.105038 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:22.104876    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:22.105038 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:22.104975    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:22.294634 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:22.294595    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="6146d350d0d0c26776ac70259d9f60a9a89c8ab6463aa9f1934b00d7c8bc2b8a" exitCode=0
Jul 23 22:23:22.294799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:22.294650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"6146d350d0d0c26776ac70259d9f60a9a89c8ab6463aa9f1934b00d7c8bc2b8a"}
Jul 23 22:23:23.104726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:23.104691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:23.104936 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.104820    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:23.726882 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:23.726844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:23.727361 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.726978    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:23:23.727361 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.727038    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs podName:ef8a2fa6-bcad-403b-a97a-bc833bd9586c nodeName:}" failed. No retries permitted until 2026-07-23 22:23:55.727025076 +0000 UTC m=+66.090691453 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs") pod "network-metrics-daemon-pzvld" (UID: "ef8a2fa6-bcad-403b-a97a-bc833bd9586c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 22:23:23.827485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:23.827410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:23.827485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:23.827448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827564    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827584    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827613    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827626    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret podName:8dbe3df2-31d4-4acf-abb7-04f4fa745718 nodeName:}" failed. No retries permitted until 2026-07-23 22:23:55.827607801 +0000 UTC m=+66.191274162 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret") pod "global-pull-secret-syncer-x5h2b" (UID: "8dbe3df2-31d4-4acf-abb7-04f4fa745718") : object "kube-system"/"original-pull-secret" not registered
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827628    2559 projected.go:196] Error preparing data for projected volume kube-api-access-hs4jg for pod openshift-network-diagnostics/network-check-target-5xft7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:23:23.827695 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:23.827684    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg podName:f874f60a-1c41-4f53-9214-70b58a76083b nodeName:}" failed. No retries permitted until 2026-07-23 22:23:55.82766865 +0000 UTC m=+66.191335026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs4jg" (UniqueName: "kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg") pod "network-check-target-5xft7" (UID: "f874f60a-1c41-4f53-9214-70b58a76083b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 22:23:24.104463 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:24.104380    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:24.104637 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:24.104519    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:24.104637 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:24.104574    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:24.104737 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:24.104683    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:25.104814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:25.104775    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:25.105439 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:25.104916    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:26.104024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:26.103943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:26.104024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:26.103983    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:26.104263 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:26.104069    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:26.104263 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:26.104218    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:27.104010 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:27.103978    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:27.104463 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:27.104082    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:27.310730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:27.310692    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="296801fb6f954d234f336f9be418c3173b6631906eabf355eea7fd4b76176d39" exitCode=0
Jul 23 22:23:27.310915 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:27.310753    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"296801fb6f954d234f336f9be418c3173b6631906eabf355eea7fd4b76176d39"}
Jul 23 22:23:28.104537 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:28.104502    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:28.105035 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:28.104620    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:28.105035 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:28.104686    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:28.105035 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:28.104807    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:28.315968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:28.315930    2559 generic.go:356] "Generic (PLEG): container finished" podID="e07a8b0e-1d52-4822-8d02-4a7f82b83486" containerID="4f3465f9a7ef8c39cd4bf18862e5eb0432038b34b3b5a46f49d2a32320ad75f4" exitCode=0
Jul 23 22:23:28.316150 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:28.316012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerDied","Data":"4f3465f9a7ef8c39cd4bf18862e5eb0432038b34b3b5a46f49d2a32320ad75f4"}
Jul 23 22:23:29.104377 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:29.104336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:29.104550 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:29.104440    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:29.324054 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:29.324021    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" event={"ID":"e07a8b0e-1d52-4822-8d02-4a7f82b83486","Type":"ContainerStarted","Data":"b0aff8fa235685d8f38cd4bd43c0ec00afc305d65d1dabe98a403906847854c3"}
Jul 23 22:23:29.348508 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:29.348456    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8pl4b" podStartSLOduration=9.674515194 podStartE2EDuration="39.348438538s" totalImagesPullingTime="29.673923344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:29.348284875 +0000 UTC m=+39.711951258" watchObservedRunningTime="2026-07-23 22:23:29.348438538 +0000 UTC m=+39.712104921"
Jul 23 22:23:30.105255 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:30.105070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:30.105453 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:30.105308    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:30.105453 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:30.105141    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:30.105453 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:30.105440    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:31.103930 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:31.103875    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:31.104321 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:31.104000    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:32.104168 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:32.104139    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:32.104543 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:32.104232    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:32.104543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:32.104327    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:32.104543 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:32.104412    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:33.103854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:33.103822    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:33.104046 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:33.103942    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:34.104565 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:34.104530    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:34.104966 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:34.104639    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:34.104966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:34.104680    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:34.104966 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:34.104736    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:35.104204 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:35.104164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:35.104383 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:35.104267    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:36.104386 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:36.104353    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:36.104957 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:36.104353    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:36.104957 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:36.104490    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:36.104957 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:36.104526    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:37.103904 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:37.103870    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:37.104076 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:37.103983    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:38.104670 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:38.104636    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:38.105167 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:38.104737    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:38.105167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:38.104813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:38.105167 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:38.104959    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:39.104678 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:39.104644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:39.105156 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:39.104755    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xft7" podUID="f874f60a-1c41-4f53-9214-70b58a76083b"
Jul 23 22:23:40.105493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.105459    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:40.105852 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:40.105553    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5h2b" podUID="8dbe3df2-31d4-4acf-abb7-04f4fa745718"
Jul 23 22:23:40.105852 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.105610    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:40.105852 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:40.105709    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pzvld" podUID="ef8a2fa6-bcad-403b-a97a-bc833bd9586c"
Jul 23 22:23:40.512808 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.512778    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeReady"
Jul 23 22:23:40.512994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.512920    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 22:23:40.587668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.587637    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r9mlv"]
Jul 23 22:23:40.600179 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.600148    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-289lh"]
Jul 23 22:23:40.600316 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.600268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.604880 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.604554    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 22:23:40.604880 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.604711    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 22:23:40.604880 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.604812    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fmr8p\""
Jul 23 22:23:40.614056 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.614035    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9mlv"]
Jul 23 22:23:40.614209 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.614192    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.616240 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.616193    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-289lh"]
Jul 23 22:23:40.617041 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617017    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2wl7f"]
Jul 23 22:23:40.617672 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617637    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 22:23:40.617796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617780    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 22:23:40.617877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617801    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 22:23:40.617877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617816    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 22:23:40.618078 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.617939    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h4r9r\""
Jul 23 22:23:40.635755 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.635726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.637324 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.637302    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2wl7f"]
Jul 23 22:23:40.638454 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.638430    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 22:23:40.638558 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.638510    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 22:23:40.638700 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.638682    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6t6v4\""
Jul 23 22:23:40.638803 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.638750    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 22:23:40.749489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a62325-5e79-423c-8be1-7f2064e1ecc1-data-volume\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.749648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/148a0ac9-e46b-4ef7-b285-278e4781d905-cert\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.749648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749573    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12618903-1464-4a3c-afd1-0d0f136489b4-tmp-dir\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.749648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a62325-5e79-423c-8be1-7f2064e1ecc1-crio-socket\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.749648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xw48\" (UniqueName: \"kubernetes.io/projected/148a0ac9-e46b-4ef7-b285-278e4781d905-kube-api-access-7xw48\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.749780 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsmpc\" (UniqueName: \"kubernetes.io/projected/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-api-access-jsmpc\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.749780 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.749780 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a62325-5e79-423c-8be1-7f2064e1ecc1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.749866 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749797    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12618903-1464-4a3c-afd1-0d0f136489b4-config-volume\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.749866 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749824    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q488\" (UniqueName: \"kubernetes.io/projected/12618903-1464-4a3c-afd1-0d0f136489b4-kube-api-access-2q488\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.749866 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.749847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12618903-1464-4a3c-afd1-0d0f136489b4-metrics-tls\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.850501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.850501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a62325-5e79-423c-8be1-7f2064e1ecc1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.850501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850481    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12618903-1464-4a3c-afd1-0d0f136489b4-config-volume\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.850720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q488\" (UniqueName: \"kubernetes.io/projected/12618903-1464-4a3c-afd1-0d0f136489b4-kube-api-access-2q488\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12618903-1464-4a3c-afd1-0d0f136489b4-metrics-tls\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.850928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a62325-5e79-423c-8be1-7f2064e1ecc1-data-volume\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/148a0ac9-e46b-4ef7-b285-278e4781d905-cert\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12618903-1464-4a3c-afd1-0d0f136489b4-tmp-dir\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a62325-5e79-423c-8be1-7f2064e1ecc1-crio-socket\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851095    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xw48\" (UniqueName: \"kubernetes.io/projected/148a0ac9-e46b-4ef7-b285-278e4781d905-kube-api-access-7xw48\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsmpc\" (UniqueName: \"kubernetes.io/projected/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-api-access-jsmpc\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12618903-1464-4a3c-afd1-0d0f136489b4-tmp-dir\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12618903-1464-4a3c-afd1-0d0f136489b4-config-volume\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851773    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a62325-5e79-423c-8be1-7f2064e1ecc1-data-volume\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.853092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.851941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a62325-5e79-423c-8be1-7f2064e1ecc1-crio-socket\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.856476 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.856453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a62325-5e79-423c-8be1-7f2064e1ecc1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.856682 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.856490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12618903-1464-4a3c-afd1-0d0f136489b4-metrics-tls\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.856682 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.856551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/148a0ac9-e46b-4ef7-b285-278e4781d905-cert\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.867631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.867609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xw48\" (UniqueName: \"kubernetes.io/projected/148a0ac9-e46b-4ef7-b285-278e4781d905-kube-api-access-7xw48\") pod \"ingress-canary-2wl7f\" (UID: \"148a0ac9-e46b-4ef7-b285-278e4781d905\") " pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:40.871119 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.871051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q488\" (UniqueName: \"kubernetes.io/projected/12618903-1464-4a3c-afd1-0d0f136489b4-kube-api-access-2q488\") pod \"dns-default-r9mlv\" (UID: \"12618903-1464-4a3c-afd1-0d0f136489b4\") " pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.872550 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.872530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsmpc\" (UniqueName: \"kubernetes.io/projected/c5a62325-5e79-423c-8be1-7f2064e1ecc1-kube-api-access-jsmpc\") pod \"insights-runtime-extractor-289lh\" (UID: \"c5a62325-5e79-423c-8be1-7f2064e1ecc1\") " pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.910066 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.910039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:40.923230 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.923202    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-289lh"
Jul 23 22:23:40.944961 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:40.944934    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2wl7f"
Jul 23 22:23:41.104931 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.104837    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:41.108859 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.108830    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-655rt\""
Jul 23 22:23:41.108859 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.108846    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 22:23:41.109302 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.109130    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 22:23:41.146824 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.146791    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2wl7f"]
Jul 23 22:23:41.149923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.149881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-289lh"]
Jul 23 22:23:41.150076 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:41.150056    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148a0ac9_e46b_4ef7_b285_278e4781d905.slice/crio-6e1d5890c932d191c643a73acb96fb1eed3d6436e1be8d48d1c6f64def38fa79 WatchSource:0}: Error finding container 6e1d5890c932d191c643a73acb96fb1eed3d6436e1be8d48d1c6f64def38fa79: Status 404 returned error can't find the container with id 6e1d5890c932d191c643a73acb96fb1eed3d6436e1be8d48d1c6f64def38fa79
Jul 23 22:23:41.152952 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:41.152929    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a62325_5e79_423c_8be1_7f2064e1ecc1.slice/crio-68e0df1fa93fd3933c1c9a2665e3b0a8ce2b98015cfc86e5d331ca02f8e24a15 WatchSource:0}: Error finding container 68e0df1fa93fd3933c1c9a2665e3b0a8ce2b98015cfc86e5d331ca02f8e24a15: Status 404 returned error can't find the container with id 68e0df1fa93fd3933c1c9a2665e3b0a8ce2b98015cfc86e5d331ca02f8e24a15
Jul 23 22:23:41.318573 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.318532    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9mlv"]
Jul 23 22:23:41.322251 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:41.322222    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12618903_1464_4a3c_afd1_0d0f136489b4.slice/crio-afb8b72926b1d69cb05392d8ed93dea0a46ec530ada6543dbfd51d95e9f6a8f2 WatchSource:0}: Error finding container afb8b72926b1d69cb05392d8ed93dea0a46ec530ada6543dbfd51d95e9f6a8f2: Status 404 returned error can't find the container with id afb8b72926b1d69cb05392d8ed93dea0a46ec530ada6543dbfd51d95e9f6a8f2
Jul 23 22:23:41.351119 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.351091    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289lh" event={"ID":"c5a62325-5e79-423c-8be1-7f2064e1ecc1","Type":"ContainerStarted","Data":"68e0df1fa93fd3933c1c9a2665e3b0a8ce2b98015cfc86e5d331ca02f8e24a15"}
Jul 23 22:23:41.352299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.352260    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9mlv" event={"ID":"12618903-1464-4a3c-afd1-0d0f136489b4","Type":"ContainerStarted","Data":"afb8b72926b1d69cb05392d8ed93dea0a46ec530ada6543dbfd51d95e9f6a8f2"}
Jul 23 22:23:41.353288 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:41.353265    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2wl7f" event={"ID":"148a0ac9-e46b-4ef7-b285-278e4781d905","Type":"ContainerStarted","Data":"6e1d5890c932d191c643a73acb96fb1eed3d6436e1be8d48d1c6f64def38fa79"}
Jul 23 22:23:42.106681 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:42.105305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:42.106681 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:42.105940    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:42.110297 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:42.109060    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2pbtn\""
Jul 23 22:23:42.110297 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:42.109283    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 22:23:42.110297 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:42.109476    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 22:23:47.374513 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:47.373996    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2wl7f" event={"ID":"148a0ac9-e46b-4ef7-b285-278e4781d905","Type":"ContainerStarted","Data":"44d563e005282f50e9b3a8401b7b386315c4f2fa71f3111498d2274db9fb6a1d"}
Jul 23 22:23:47.379622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:47.379581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289lh" event={"ID":"c5a62325-5e79-423c-8be1-7f2064e1ecc1","Type":"ContainerStarted","Data":"2db9d7a79be6246b7640da10df7477e9e845523bd09a42deb2a22466700b55a0"}
Jul 23 22:23:47.382214 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:47.382177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9mlv" event={"ID":"12618903-1464-4a3c-afd1-0d0f136489b4","Type":"ContainerStarted","Data":"bc372d7e94113461bba0c5570bc147d1c3d7791e10b90d7d941439c327cc28d2"}
Jul 23 22:23:48.386673 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:48.386631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9mlv" event={"ID":"12618903-1464-4a3c-afd1-0d0f136489b4","Type":"ContainerStarted","Data":"f49548b7020503569a5981ee7c70eaa80d3f5938c530f33e0b7832023870a4d9"}
Jul 23 22:23:48.413000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:48.412940    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r9mlv" podStartSLOduration=2.502283452 podStartE2EDuration="8.412917133s" totalImagesPullingTime="5.910633681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:48.412673863 +0000 UTC m=+58.776340246" watchObservedRunningTime="2026-07-23 22:23:48.412917133 +0000 UTC m=+58.776583516"
Jul 23 22:23:48.413203 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:48.413059    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2wl7f" podStartSLOduration=2.329308874 podStartE2EDuration="8.413050444s" totalImagesPullingTime="6.08374157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:47.396738665 +0000 UTC m=+57.760405038" watchObservedRunningTime="2026-07-23 22:23:48.413050444 +0000 UTC m=+58.776716830"
Jul 23 22:23:49.389806 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:49.389773    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:23:50.299809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:50.299775    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gnptj"
Jul 23 22:23:51.397131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:51.397087    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289lh" event={"ID":"c5a62325-5e79-423c-8be1-7f2064e1ecc1","Type":"ContainerStarted","Data":"3b5bd1be15c319a9a58bae82e5ed2071c9f2821f144e951c901473db7b0288f9"}
Jul 23 22:23:53.404352 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.404313    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"]
Jul 23 22:23:53.418367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.418304    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"]
Jul 23 22:23:53.418524 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.418422    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.421349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.421324    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 22:23:53.422184 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.422161    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 22:23:53.422387 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.422370    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ktf5s\""
Jul 23 22:23:53.422629 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.422608    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 22:23:53.422793 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.422778    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 22:23:53.422964 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.422948    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 22:23:53.451990 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.451942    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.452137 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.451999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.452137 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.452055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47j4\" (UniqueName: \"kubernetes.io/projected/908418ef-7e8b-4b18-a63e-b23d6ab69739-kube-api-access-m47j4\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.452137 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.452097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/908418ef-7e8b-4b18-a63e-b23d6ab69739-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.553484 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.553448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.553484 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.553489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.553727 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.553515    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47j4\" (UniqueName: \"kubernetes.io/projected/908418ef-7e8b-4b18-a63e-b23d6ab69739-kube-api-access-m47j4\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.553727 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.553545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/908418ef-7e8b-4b18-a63e-b23d6ab69739-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.553727 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:53.553627    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 22:23:53.553727 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:23:53.553708    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls podName:908418ef-7e8b-4b18-a63e-b23d6ab69739 nodeName:}" failed. No retries permitted until 2026-07-23 22:23:54.053687301 +0000 UTC m=+64.417353685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-d5ztq" (UID: "908418ef-7e8b-4b18-a63e-b23d6ab69739") : secret "prometheus-operator-tls" not found
Jul 23 22:23:53.554329 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.554302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/908418ef-7e8b-4b18-a63e-b23d6ab69739-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.556229 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.556205    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:53.563379 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:53.563356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47j4\" (UniqueName: \"kubernetes.io/projected/908418ef-7e8b-4b18-a63e-b23d6ab69739-kube-api-access-m47j4\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:54.056888 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.056855    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:54.059402 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.059369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/908418ef-7e8b-4b18-a63e-b23d6ab69739-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-d5ztq\" (UID: \"908418ef-7e8b-4b18-a63e-b23d6ab69739\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:54.224820 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.224779    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-7p69h"]
Jul 23 22:23:54.227682 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.227666    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:23:54.230756 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.230731    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 22:23:54.230882 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.230778    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jtq76\""
Jul 23 22:23:54.230882 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.230785    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 22:23:54.249239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.249220    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7p69h"]
Jul 23 22:23:54.258665 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.258645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg87c\" (UniqueName: \"kubernetes.io/projected/c6a4b1ef-5229-4d78-a855-1c3fe7753f73-kube-api-access-jg87c\") pod \"downloads-5bc67dcd46-7p69h\" (UID: \"c6a4b1ef-5229-4d78-a855-1c3fe7753f73\") " pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:23:54.330302 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.330214    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"
Jul 23 22:23:54.359569 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.359529    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg87c\" (UniqueName: \"kubernetes.io/projected/c6a4b1ef-5229-4d78-a855-1c3fe7753f73-kube-api-access-jg87c\") pod \"downloads-5bc67dcd46-7p69h\" (UID: \"c6a4b1ef-5229-4d78-a855-1c3fe7753f73\") " pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:23:54.371121 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.371091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg87c\" (UniqueName: \"kubernetes.io/projected/c6a4b1ef-5229-4d78-a855-1c3fe7753f73-kube-api-access-jg87c\") pod \"downloads-5bc67dcd46-7p69h\" (UID: \"c6a4b1ef-5229-4d78-a855-1c3fe7753f73\") " pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:23:54.409650 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.409408    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289lh" event={"ID":"c5a62325-5e79-423c-8be1-7f2064e1ecc1","Type":"ContainerStarted","Data":"64e0381e361a916ddfae64eb65aff5f88aaf5542e9f07abc87f964f0b10ce209"}
Jul 23 22:23:54.430406 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.430338    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-289lh" podStartSLOduration=1.6607225300000001 podStartE2EDuration="14.430317286s" totalImagesPullingTime="12.769594756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:23:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:54.42994983 +0000 UTC m=+64.793616239" watchObservedRunningTime="2026-07-23 22:23:54.430317286 +0000 UTC m=+64.793983670"
Jul 23 22:23:54.457907 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.457874    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq"]
Jul 23 22:23:54.461640 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:54.461613    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908418ef_7e8b_4b18_a63e_b23d6ab69739.slice/crio-07a132baf17accf57d43fc10d9d5f384c263e31c5b5fb4b79841d38890feb9d8 WatchSource:0}: Error finding container 07a132baf17accf57d43fc10d9d5f384c263e31c5b5fb4b79841d38890feb9d8: Status 404 returned error can't find the container with id 07a132baf17accf57d43fc10d9d5f384c263e31c5b5fb4b79841d38890feb9d8
Jul 23 22:23:54.536046 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.536010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:23:54.662871 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:54.662839    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7p69h"]
Jul 23 22:23:54.665689 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:54.665652    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a4b1ef_5229_4d78_a855_1c3fe7753f73.slice/crio-648a369809e9fca3146d988b6c8239a9749f0f9dfa8b8748a770fe7e4527b4f3 WatchSource:0}: Error finding container 648a369809e9fca3146d988b6c8239a9749f0f9dfa8b8748a770fe7e4527b4f3: Status 404 returned error can't find the container with id 648a369809e9fca3146d988b6c8239a9749f0f9dfa8b8748a770fe7e4527b4f3
Jul 23 22:23:55.415354 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.415311    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq" event={"ID":"908418ef-7e8b-4b18-a63e-b23d6ab69739","Type":"ContainerStarted","Data":"07a132baf17accf57d43fc10d9d5f384c263e31c5b5fb4b79841d38890feb9d8"}
Jul 23 22:23:55.416736 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.416706    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7p69h" event={"ID":"c6a4b1ef-5229-4d78-a855-1c3fe7753f73","Type":"ContainerStarted","Data":"648a369809e9fca3146d988b6c8239a9749f0f9dfa8b8748a770fe7e4527b4f3"}
Jul 23 22:23:55.768536 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.768503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:55.771200 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.771168    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 22:23:55.782495 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.782470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef8a2fa6-bcad-403b-a97a-bc833bd9586c-metrics-certs\") pod \"network-metrics-daemon-pzvld\" (UID: \"ef8a2fa6-bcad-403b-a97a-bc833bd9586c\") " pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:55.869511 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.869474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:55.869666 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.869525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:55.872046 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.872017    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 22:23:55.872173 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.872016    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 22:23:55.882869 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.882850    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 22:23:55.883168 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.883146    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8dbe3df2-31d4-4acf-abb7-04f4fa745718-original-pull-secret\") pod \"global-pull-secret-syncer-x5h2b\" (UID: \"8dbe3df2-31d4-4acf-abb7-04f4fa745718\") " pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:55.893646 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.893618    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs4jg\" (UniqueName: \"kubernetes.io/projected/f874f60a-1c41-4f53-9214-70b58a76083b-kube-api-access-hs4jg\") pod \"network-check-target-5xft7\" (UID: \"f874f60a-1c41-4f53-9214-70b58a76083b\") " pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:55.922583 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.922545    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5h2b"
Jul 23 22:23:55.936664 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.936575    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2pbtn\""
Jul 23 22:23:55.940446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:55.940302    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pzvld"
Jul 23 22:23:56.086692 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.086633    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5h2b"]
Jul 23 22:23:56.089572 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:56.089532    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbe3df2_31d4_4acf_abb7_04f4fa745718.slice/crio-e29d748457c0c12d1dd2c69f2e23fe6631f15cf1c422f720360ac2aeca36d9ec WatchSource:0}: Error finding container e29d748457c0c12d1dd2c69f2e23fe6631f15cf1c422f720360ac2aeca36d9ec: Status 404 returned error can't find the container with id e29d748457c0c12d1dd2c69f2e23fe6631f15cf1c422f720360ac2aeca36d9ec
Jul 23 22:23:56.103746 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.103718    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pzvld"]
Jul 23 22:23:56.107401 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:56.107369    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8a2fa6_bcad_403b_a97a_bc833bd9586c.slice/crio-0550e87d3671b0a62094cbb60bf87e64687e028e73deb22fb1c59c2aecc3acd2 WatchSource:0}: Error finding container 0550e87d3671b0a62094cbb60bf87e64687e028e73deb22fb1c59c2aecc3acd2: Status 404 returned error can't find the container with id 0550e87d3671b0a62094cbb60bf87e64687e028e73deb22fb1c59c2aecc3acd2
Jul 23 22:23:56.117353 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.117332    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-655rt\""
Jul 23 22:23:56.125724 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.125687    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:23:56.265613 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.265581    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5xft7"]
Jul 23 22:23:56.268578 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:23:56.268549    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf874f60a_1c41_4f53_9214_70b58a76083b.slice/crio-f5709b664b4c9261438489b8ec4bd0446d4944de50b748283f68f8d45be2e708 WatchSource:0}: Error finding container f5709b664b4c9261438489b8ec4bd0446d4944de50b748283f68f8d45be2e708: Status 404 returned error can't find the container with id f5709b664b4c9261438489b8ec4bd0446d4944de50b748283f68f8d45be2e708
Jul 23 22:23:56.422191 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.422116    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5h2b" event={"ID":"8dbe3df2-31d4-4acf-abb7-04f4fa745718","Type":"ContainerStarted","Data":"e29d748457c0c12d1dd2c69f2e23fe6631f15cf1c422f720360ac2aeca36d9ec"}
Jul 23 22:23:56.425024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.424428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq" event={"ID":"908418ef-7e8b-4b18-a63e-b23d6ab69739","Type":"ContainerStarted","Data":"e8b6e2ddc89d8419bb32936a8423137bf23d9575219fc0e08ed4f8d0f911aad0"}
Jul 23 22:23:56.425024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.424486    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq" event={"ID":"908418ef-7e8b-4b18-a63e-b23d6ab69739","Type":"ContainerStarted","Data":"f46976b3b5bcf6572950e3f7860d86960f4d5dda0157b61d6f023a7048551ee1"}
Jul 23 22:23:56.426568 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.426531    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5xft7" event={"ID":"f874f60a-1c41-4f53-9214-70b58a76083b","Type":"ContainerStarted","Data":"f5709b664b4c9261438489b8ec4bd0446d4944de50b748283f68f8d45be2e708"}
Jul 23 22:23:56.428325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.428300    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzvld" event={"ID":"ef8a2fa6-bcad-403b-a97a-bc833bd9586c","Type":"ContainerStarted","Data":"0550e87d3671b0a62094cbb60bf87e64687e028e73deb22fb1c59c2aecc3acd2"}
Jul 23 22:23:56.445919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:56.445837    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-d5ztq" podStartSLOduration=2.229335837 podStartE2EDuration="3.4458155s" totalImagesPullingTime="1.216479663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:56.444342309 +0000 UTC m=+66.808008705" watchObservedRunningTime="2026-07-23 22:23:56.4458155 +0000 UTC m=+66.809481917"
Jul 23 22:23:57.438968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:57.438935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzvld" event={"ID":"ef8a2fa6-bcad-403b-a97a-bc833bd9586c","Type":"ContainerStarted","Data":"431188055b1dc911521927df8d06146047676d84e0f99b34c3f31a317363cf14"}
Jul 23 22:23:58.446567 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.446526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pzvld" event={"ID":"ef8a2fa6-bcad-403b-a97a-bc833bd9586c","Type":"ContainerStarted","Data":"c9e9792c55df8151796b72d80859293c1c1428fe90293f079db6e34db137dff5"}
Jul 23 22:23:58.469799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.469377    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pzvld" podStartSLOduration=67.272340566 podStartE2EDuration="1m8.469356007s" totalImagesPullingTime="1.197015441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:58.468428441 +0000 UTC m=+68.832094824" watchObservedRunningTime="2026-07-23 22:23:58.469356007 +0000 UTC m=+68.833022391"
Jul 23 22:23:58.871698 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.870508    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rc9kp"]
Jul 23 22:23:58.889330 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.889288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.893434 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.893402    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 22:23:58.893571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.893551    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 22:23:58.893923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.893888    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9zrvx\""
Jul 23 22:23:58.894937 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.894914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4fsn\" (UniqueName: \"kubernetes.io/projected/b42c2bf4-7109-4684-b164-bc25e8c65155-kube-api-access-n4fsn\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.894969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-metrics-client-ca\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-sys\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895041    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-root\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895077    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-textfile\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-tls\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-wtmp\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.895233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.896016 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.895707    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 22:23:58.996320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-textfile\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-tls\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-wtmp\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4fsn\" (UniqueName: \"kubernetes.io/projected/b42c2bf4-7109-4684-b164-bc25e8c65155-kube-api-access-n4fsn\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-metrics-client-ca\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996783 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-sys\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996783 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-root\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996783 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-root\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.996783 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-textfile\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.997007 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.996797    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-wtmp\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.997249 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.997223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b42c2bf4-7109-4684-b164-bc25e8c65155-sys\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.997444 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.997400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:58.997564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.997447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b42c2bf4-7109-4684-b164-bc25e8c65155-metrics-client-ca\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:59.000004 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:58.999982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:59.000157 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:59.000132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b42c2bf4-7109-4684-b164-bc25e8c65155-node-exporter-tls\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:59.011150 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:59.011125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4fsn\" (UniqueName: \"kubernetes.io/projected/b42c2bf4-7109-4684-b164-bc25e8c65155-kube-api-access-n4fsn\") pod \"node-exporter-rc9kp\" (UID: \"b42c2bf4-7109-4684-b164-bc25e8c65155\") " pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:59.201986 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:59.201887    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rc9kp"
Jul 23 22:23:59.395121 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:23:59.395085    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r9mlv"
Jul 23 22:24:00.789533 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.789502    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8589f78d67-q2ncd"]
Jul 23 22:24:00.792586 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.792278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.795631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.795610    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 22:24:00.795758 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.795735    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 22:24:00.795852 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.795836    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5mpctmua6ond1\""
Jul 23 22:24:00.796010 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.795987    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 22:24:00.796205 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.796188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 22:24:00.798741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.798722    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 22:24:00.799327 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.799307    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mm874\""
Jul 23 22:24:00.811290 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811262    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8589f78d67-q2ncd"]
Jul 23 22:24:00.811594 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811703    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc2563dd-35e1-4328-b8db-827c3c353f6b-metrics-client-ca\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-grpc-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.811995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.811847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw2kd\" (UniqueName: \"kubernetes.io/projected/bc2563dd-35e1-4328-b8db-827c3c353f6b-kube-api-access-cw2kd\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.912715 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.912674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-grpc-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.912915 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.912733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.912988 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.912949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw2kd\" (UniqueName: \"kubernetes.io/projected/bc2563dd-35e1-4328-b8db-827c3c353f6b-kube-api-access-cw2kd\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.913045 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.913010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.913106 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.913047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.913106 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.913075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.913559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.913531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc2563dd-35e1-4328-b8db-827c3c353f6b-metrics-client-ca\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.913641 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.913599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.914720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.914689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc2563dd-35e1-4328-b8db-827c3c353f6b-metrics-client-ca\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.916313 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.916288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.916681 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.916526    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-grpc-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.917061 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.917018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-tls\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.917061 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.917051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.917999 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.917974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.918412 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.918386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc2563dd-35e1-4328-b8db-827c3c353f6b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:00.923111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:00.923085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw2kd\" (UniqueName: \"kubernetes.io/projected/bc2563dd-35e1-4328-b8db-827c3c353f6b-kube-api-access-cw2kd\") pod \"thanos-querier-8589f78d67-q2ncd\" (UID: \"bc2563dd-35e1-4328-b8db-827c3c353f6b\") " pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:01.105273 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:01.105187    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:01.803818 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:24:01.803761    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42c2bf4_7109_4684_b164_bc25e8c65155.slice/crio-31ef5e7311594298342dd8088f6cd44ad1a97184d5064d11658ec21f1b1ee7bf WatchSource:0}: Error finding container 31ef5e7311594298342dd8088f6cd44ad1a97184d5064d11658ec21f1b1ee7bf: Status 404 returned error can't find the container with id 31ef5e7311594298342dd8088f6cd44ad1a97184d5064d11658ec21f1b1ee7bf
Jul 23 22:24:01.961849 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:01.961809    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8589f78d67-q2ncd"]
Jul 23 22:24:01.964356 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:24:01.964322    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2563dd_35e1_4328_b8db_827c3c353f6b.slice/crio-b8601f18bbe8c71c38e244ab437195bd7c3a3c8c1ccefed360b06e73261583fc WatchSource:0}: Error finding container b8601f18bbe8c71c38e244ab437195bd7c3a3c8c1ccefed360b06e73261583fc: Status 404 returned error can't find the container with id b8601f18bbe8c71c38e244ab437195bd7c3a3c8c1ccefed360b06e73261583fc
Jul 23 22:24:02.465875 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.465410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5xft7" event={"ID":"f874f60a-1c41-4f53-9214-70b58a76083b","Type":"ContainerStarted","Data":"83199db6b1be13362ad710813e167a71453bc2afea6693b3912698fc83c5e204"}
Jul 23 22:24:02.465875 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.465511    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:24:02.467713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.467656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc9kp" event={"ID":"b42c2bf4-7109-4684-b164-bc25e8c65155","Type":"ContainerStarted","Data":"31ef5e7311594298342dd8088f6cd44ad1a97184d5064d11658ec21f1b1ee7bf"}
Jul 23 22:24:02.469276 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.469247    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"b8601f18bbe8c71c38e244ab437195bd7c3a3c8c1ccefed360b06e73261583fc"}
Jul 23 22:24:02.471736 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.471706    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5h2b" event={"ID":"8dbe3df2-31d4-4acf-abb7-04f4fa745718","Type":"ContainerStarted","Data":"8c2654f0d297632f65c2ac2505f88b8858b4b24fa570147f94a43e6cf96c4c35"}
Jul 23 22:24:02.486946 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.485880    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5xft7" podStartSLOduration=66.952299879 podStartE2EDuration="1m12.485866243s" totalImagesPullingTime="5.533566364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:02.485304514 +0000 UTC m=+72.848970897" watchObservedRunningTime="2026-07-23 22:24:02.485866243 +0000 UTC m=+72.849532617"
Jul 23 22:24:02.504135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:02.503595    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x5h2b" podStartSLOduration=66.776164128 podStartE2EDuration="1m12.503573997s" totalImagesPullingTime="5.727409869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:02.501809128 +0000 UTC m=+72.865475513" watchObservedRunningTime="2026-07-23 22:24:02.503573997 +0000 UTC m=+72.867240379"
Jul 23 22:24:03.339776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.339733    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:03.343340 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.343308    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.345733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.345528    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 22:24:03.346581 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.346560    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 22:24:03.346754 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.346738    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 22:24:03.346948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.346924    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 22:24:03.348876 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.348646    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-djwzz\""
Jul 23 22:24:03.348876 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.348704    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 22:24:03.362968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.362874    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:03.368422 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.368393    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5549556469-s828j"]
Jul 23 22:24:03.371609 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.371583    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.374549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.374379    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 22:24:03.374549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.374419    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ldsqt\""
Jul 23 22:24:03.374549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.374429    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 22:24:03.374549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.374473    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6bg5skpn74al0\""
Jul 23 22:24:03.374822 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.374701    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 22:24:03.377802 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.377782    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 22:24:03.389478 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.389418    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5549556469-s828j"]
Jul 23 22:24:03.434194 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45567508-5e3d-4f78-9d04-00d6801a6bc4-audit-log\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434217    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-client-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434368    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434389    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-tls\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-client-certs\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434501    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-metrics-server-audit-profiles\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.434579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434565    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434815 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l594\" (UniqueName: \"kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.434815 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.434618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2bgk\" (UniqueName: \"kubernetes.io/projected/45567508-5e3d-4f78-9d04-00d6801a6bc4-kube-api-access-f2bgk\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.481582 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.481177    2559 generic.go:356] "Generic (PLEG): container finished" podID="b42c2bf4-7109-4684-b164-bc25e8c65155" containerID="3000075cc585af6f6a4782819088684b8cf331844136d8ccf08648f24c6728ff" exitCode=0
Jul 23 22:24:03.481582 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.481440    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc9kp" event={"ID":"b42c2bf4-7109-4684-b164-bc25e8c65155","Type":"ContainerDied","Data":"3000075cc585af6f6a4782819088684b8cf331844136d8ccf08648f24c6728ff"}
Jul 23 22:24:03.535497 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535465    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.535497 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-metrics-server-audit-profiles\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l594\" (UniqueName: \"kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2bgk\" (UniqueName: \"kubernetes.io/projected/45567508-5e3d-4f78-9d04-00d6801a6bc4-kube-api-access-f2bgk\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45567508-5e3d-4f78-9d04-00d6801a6bc4-audit-log\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.535745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535735    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.536128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-client-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.536128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.536128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535786    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-tls\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.536128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.535813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-client-certs\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.537269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.537322    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"]
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.537564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.537934    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.538351    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.539286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.539244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/45567508-5e3d-4f78-9d04-00d6801a6bc4-metrics-server-audit-profiles\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.539683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.539549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.540388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.539928    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/45567508-5e3d-4f78-9d04-00d6801a6bc4-audit-log\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.540388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.540347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-client-certs\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.540882 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.540850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-secret-metrics-server-tls\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.541473 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.541448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45567508-5e3d-4f78-9d04-00d6801a6bc4-client-ca-bundle\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.542165 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.542138    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:03.542165 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.542158    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.548551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.548494    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 22:24:03.549508 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.548887    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xvvxd\""
Jul 23 22:24:03.558514 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.558491    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"]
Jul 23 22:24:03.561140 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.561111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l594\" (UniqueName: \"kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594\") pod \"console-6dbf777bdf-zhtmj\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") " pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.561854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.561809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2bgk\" (UniqueName: \"kubernetes.io/projected/45567508-5e3d-4f78-9d04-00d6801a6bc4-kube-api-access-f2bgk\") pod \"metrics-server-5549556469-s828j\" (UID: \"45567508-5e3d-4f78-9d04-00d6801a6bc4\") " pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.636705 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.636596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:03.636873 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.636788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwsm8\" (UniqueName: \"kubernetes.io/projected/1e7537ff-55d1-4e98-840f-728ec741c417-kube-api-access-fwsm8\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:03.655639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.655601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:03.683123 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.683088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:03.737743 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.737700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwsm8\" (UniqueName: \"kubernetes.io/projected/1e7537ff-55d1-4e98-840f-728ec741c417-kube-api-access-fwsm8\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:03.737963 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.737753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:03.737963 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:24:03.737883    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 23 22:24:03.738093 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:24:03.737997    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert podName:1e7537ff-55d1-4e98-840f-728ec741c417 nodeName:}" failed. No retries permitted until 2026-07-23 22:24:04.237975601 +0000 UTC m=+74.601641977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-kqkm5" (UID: "1e7537ff-55d1-4e98-840f-728ec741c417") : secret "monitoring-plugin-cert" not found
Jul 23 22:24:03.748538 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:03.748509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwsm8\" (UniqueName: \"kubernetes.io/projected/1e7537ff-55d1-4e98-840f-728ec741c417-kube-api-access-fwsm8\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:04.076125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.076076    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:04.092329 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.092263    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5549556469-s828j"]
Jul 23 22:24:04.243196 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.243164    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:04.246196 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.246143    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e7537ff-55d1-4e98-840f-728ec741c417-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kqkm5\" (UID: \"1e7537ff-55d1-4e98-840f-728ec741c417\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:04.475635 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.475596    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:04.489483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.489451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc9kp" event={"ID":"b42c2bf4-7109-4684-b164-bc25e8c65155","Type":"ContainerStarted","Data":"e4c16adcf804d81888590c0535cadcf6bb6ff0fe51a8f121331a73224b04f0b9"}
Jul 23 22:24:04.489612 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.489503    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc9kp" event={"ID":"b42c2bf4-7109-4684-b164-bc25e8c65155","Type":"ContainerStarted","Data":"58dade76fc6bba1c785bab9cac70246d737db46a48cda270c429d5f82508d698"}
Jul 23 22:24:04.491498 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.491465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5549556469-s828j" event={"ID":"45567508-5e3d-4f78-9d04-00d6801a6bc4","Type":"ContainerStarted","Data":"c9295d4d42eb1fad3c5c2f2e21aa37aff771ea166682bdb302e6cc2e24c8947c"}
Jul 23 22:24:04.495251 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.495095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"fd8f699ee7bccf99c34a4ba67ec31f1e4bf7a98a8df85d35e1a5cf30436fcc41"}
Jul 23 22:24:04.495251 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.495163    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"21a399318f837fb9f543d2ab223fd605f6a09f2c056a5f8c20fd69dc173d4c5a"}
Jul 23 22:24:04.495251 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.495206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"c0c469855ab974e7e3e223c1e30c8fc369c6f5c762c78b49c38c58b29a50dbad"}
Jul 23 22:24:04.496677 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.496629    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf777bdf-zhtmj" event={"ID":"08f36af2-5036-47c1-a42a-80b7c0dfd769","Type":"ContainerStarted","Data":"ba37ff92231aaba59312f35c58bc9def9883d79c33c469e3ee199b71330496e9"}
Jul 23 22:24:04.512776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.512268    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rc9kp" podStartSLOduration=5.305066313 podStartE2EDuration="6.512248506s" totalImagesPullingTime="1.120798879s" totalInitContainerRuntime="86.383314ms" isStatefulPod=true podCreationTimestamp="2026-07-23 22:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:04.510529801 +0000 UTC m=+74.874196185" watchObservedRunningTime="2026-07-23 22:24:04.512248506 +0000 UTC m=+74.875914890"
Jul 23 22:24:04.662326 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:04.662208    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"]
Jul 23 22:24:04.671863 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:24:04.671832    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7537ff_55d1_4e98_840f_728ec741c417.slice/crio-3079171a43e2e766666c7ae87feba5fe981c3965d0599ee77ed2592fc68dfaa6 WatchSource:0}: Error finding container 3079171a43e2e766666c7ae87feba5fe981c3965d0599ee77ed2592fc68dfaa6: Status 404 returned error can't find the container with id 3079171a43e2e766666c7ae87feba5fe981c3965d0599ee77ed2592fc68dfaa6
Jul 23 22:24:05.093164 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.092724    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:24:05.097375 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.097298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.103287 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.103259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 22:24:05.103543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.103521    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 22:24:05.103769 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.103749    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 22:24:05.105451 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.105122    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 22:24:05.105451 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.105285    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n92zj\""
Jul 23 22:24:05.105451 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.105347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 22:24:05.105695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.105592    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.105879    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.106178    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.106395    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.106578    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.106790    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 22:24:05.106926 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.106844    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 22:24:05.107234 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.107084    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 22:24:05.107234 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.107195    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 22:24:05.108186 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.108088    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-diqrcerm8me43\""
Jul 23 22:24:05.119248 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.119204    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151198    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-922zb\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151398    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.151493 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.151455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.154979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.155977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.155299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.255942 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.255912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.255962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.255998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-922zb\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256250    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256272    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256332    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256424    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.256725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.256489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.259717 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.257544    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.259717 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.259066    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.259717 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.259354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.260828 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.260788    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.262002 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.261823    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.262443 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.262416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.265988 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.265184    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.266637 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.266584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.271582 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.271557    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.272284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.272336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.272687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.272723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.272834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.273091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.273263    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.273564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.273697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.273612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.274619 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.274578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.275084 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.275061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-922zb\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb\") pod \"prometheus-k8s-0\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.417478 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.417174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:05.509154 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.509090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"1bdc6fbb98adb23a2d641042d4c117d3a442d27bf3fca0c49e63e7e0f9ac480d"}
Jul 23 22:24:05.509622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.509215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"93c5e547828b6b7f80f690fd54877f830fb472653ccd50721225d8a6cfb43bf4"}
Jul 23 22:24:05.509622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.509261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" event={"ID":"bc2563dd-35e1-4328-b8db-827c3c353f6b","Type":"ContainerStarted","Data":"4e298dea6bf0aa11b2fefbca1f2711eac740757cf24d6990f84169f40832482c"}
Jul 23 22:24:05.510064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.509976    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:05.515618 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.515557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5" event={"ID":"1e7537ff-55d1-4e98-840f-728ec741c417","Type":"ContainerStarted","Data":"3079171a43e2e766666c7ae87feba5fe981c3965d0599ee77ed2592fc68dfaa6"}
Jul 23 22:24:05.548819 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:05.547331    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd" podStartSLOduration=2.654769104 podStartE2EDuration="5.54730914s" totalImagesPullingTime="2.892540036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:24:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:05.545361658 +0000 UTC m=+75.909028067" watchObservedRunningTime="2026-07-23 22:24:05.54730914 +0000 UTC m=+75.910975525"
Jul 23 22:24:11.527149 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:11.527018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8589f78d67-q2ncd"
Jul 23 22:24:15.763923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:15.763811    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:24:15.766224 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:24:15.766199    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af60bac_6d3f_4425_a992_8677bafa2636.slice/crio-0fd9cb8716064bc56e13263d2e7be8a5f4d1ea31b50c888f00a31f2e069c03d9 WatchSource:0}: Error finding container 0fd9cb8716064bc56e13263d2e7be8a5f4d1ea31b50c888f00a31f2e069c03d9: Status 404 returned error can't find the container with id 0fd9cb8716064bc56e13263d2e7be8a5f4d1ea31b50c888f00a31f2e069c03d9
Jul 23 22:24:16.562690 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.561686    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7p69h" event={"ID":"c6a4b1ef-5229-4d78-a855-1c3fe7753f73","Type":"ContainerStarted","Data":"a6736139755b170a491319e19bd762521c93df2e4142cb1872ebd7666d12cde3"}
Jul 23 22:24:16.563015 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.562783    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:24:16.565746 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.565174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5549556469-s828j" event={"ID":"45567508-5e3d-4f78-9d04-00d6801a6bc4","Type":"ContainerStarted","Data":"23d97b221a0295d6bc9fb81ffa807b936c2167b0ebd8e0ef1ec22a06241a4c08"}
Jul 23 22:24:16.567383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.567283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"0fd9cb8716064bc56e13263d2e7be8a5f4d1ea31b50c888f00a31f2e069c03d9"}
Jul 23 22:24:16.570114 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.569465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf777bdf-zhtmj" event={"ID":"08f36af2-5036-47c1-a42a-80b7c0dfd769","Type":"ContainerStarted","Data":"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"}
Jul 23 22:24:16.573532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.572738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5" event={"ID":"1e7537ff-55d1-4e98-840f-728ec741c417","Type":"ContainerStarted","Data":"2a2149fd5845d39b78aa8a7ab2c2ded181d180f6e976cd2de8b9a8a8b30c3f55"}
Jul 23 22:24:16.573532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.573475    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:16.580816 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.580700    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-7p69h"
Jul 23 22:24:16.580988 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.580972    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5"
Jul 23 22:24:16.586881 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.586649    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-7p69h" podStartSLOduration=1.589030891 podStartE2EDuration="22.586632755s" totalImagesPullingTime="20.997601864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:16.585660518 +0000 UTC m=+86.949326918" watchObservedRunningTime="2026-07-23 22:24:16.586632755 +0000 UTC m=+86.950299140"
Jul 23 22:24:16.627674 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.627298    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kqkm5" podStartSLOduration=2.689124798 podStartE2EDuration="13.627277109s" totalImagesPullingTime="10.938152311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:16.626819264 +0000 UTC m=+86.990485661" watchObservedRunningTime="2026-07-23 22:24:16.627277109 +0000 UTC m=+86.990943495"
Jul 23 22:24:16.627924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.627869    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dbf777bdf-zhtmj" podStartSLOduration=2.102304708 podStartE2EDuration="13.627857323s" totalImagesPullingTime="11.525552615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:16.604306948 +0000 UTC m=+86.967973349" watchObservedRunningTime="2026-07-23 22:24:16.627857323 +0000 UTC m=+86.991523707"
Jul 23 22:24:16.669941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:16.669754    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5549556469-s828j" podStartSLOduration=2.157915382 podStartE2EDuration="13.669732374s" totalImagesPullingTime="11.511816992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:16.668665357 +0000 UTC m=+87.032331741" watchObservedRunningTime="2026-07-23 22:24:16.669732374 +0000 UTC m=+87.033398758"
Jul 23 22:24:17.579781 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:17.579722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"}
Jul 23 22:24:18.585265 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:18.585223    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b" exitCode=0
Jul 23 22:24:18.585785 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:18.585320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"}
Jul 23 22:24:23.331800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.331722    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:23.614065 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.613976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"}
Jul 23 22:24:23.614065 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.614057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"}
Jul 23 22:24:23.614349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.614085    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"}
Jul 23 22:24:23.614349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.614124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"}
Jul 23 22:24:23.614349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.614161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"}
Jul 23 22:24:23.614349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.614195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerStarted","Data":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"}
Jul 23 22:24:23.649966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.649888    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=13.071939359 podStartE2EDuration="18.649868202s" totalImagesPullingTime="5.559954492s" totalInitContainerRuntime="17.974351ms" isStatefulPod=false podCreationTimestamp="2026-07-23 22:24:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:24:23.647935766 +0000 UTC m=+94.011602171" watchObservedRunningTime="2026-07-23 22:24:23.649868202 +0000 UTC m=+94.013534585"
Jul 23 22:24:23.657071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.657044    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:23.683399 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.683364    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:23.683575 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:23.683428    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:25.417998 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:25.417964    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:24:33.484589 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:33.484553    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5xft7"
Jul 23 22:24:43.691376 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:43.691345    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:43.695385 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:43.695357    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5549556469-s828j"
Jul 23 22:24:48.355566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.355503    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dbf777bdf-zhtmj" podUID="08f36af2-5036-47c1-a42a-80b7c0dfd769" containerName="console" containerID="cri-o://5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1" gracePeriod=15
Jul 23 22:24:48.595363 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.595338    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:48.643734 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643652    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.643734 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643708    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.643734 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643732    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l594\" (UniqueName: \"kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.644012 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643767    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.644012 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643795    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.644012 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.643840    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca\") pod \"08f36af2-5036-47c1-a42a-80b7c0dfd769\" (UID: \"08f36af2-5036-47c1-a42a-80b7c0dfd769\") "
Jul 23 22:24:48.644255 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.644227    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config" (OuterVolumeSpecName: "console-config") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:24:48.644400 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.644306    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca" (OuterVolumeSpecName: "service-ca") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:24:48.644400 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.644322    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:24:48.646177 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.646156    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:24:48.646177 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.646159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594" (OuterVolumeSpecName: "kube-api-access-8l594") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "kube-api-access-8l594". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:24:48.646296 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.646234    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "08f36af2-5036-47c1-a42a-80b7c0dfd769" (UID: "08f36af2-5036-47c1-a42a-80b7c0dfd769"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:24:48.712699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.712662    2559 generic.go:356] "Generic (PLEG): container finished" podID="08f36af2-5036-47c1-a42a-80b7c0dfd769" containerID="5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1" exitCode=0
Jul 23 22:24:48.712969 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.712714    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf777bdf-zhtmj"
Jul 23 22:24:48.712969 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.712743    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf777bdf-zhtmj" event={"ID":"08f36af2-5036-47c1-a42a-80b7c0dfd769","Type":"ContainerDied","Data":"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"}
Jul 23 22:24:48.712969 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.712792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf777bdf-zhtmj" event={"ID":"08f36af2-5036-47c1-a42a-80b7c0dfd769","Type":"ContainerDied","Data":"ba37ff92231aaba59312f35c58bc9def9883d79c33c469e3ee199b71330496e9"}
Jul 23 22:24:48.712969 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.712811    2559 scope.go:117] "RemoveContainer" containerID="5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"
Jul 23 22:24:48.721483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.721465    2559 scope.go:117] "RemoveContainer" containerID="5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"
Jul 23 22:24:48.721750 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:24:48.721709    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1\": container with ID starting with 5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1 not found: ID does not exist" containerID="5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"
Jul 23 22:24:48.721796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.721760    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1"} err="failed to get container status \"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1\": rpc error: code = NotFound desc = could not find container \"5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1\": container with ID starting with 5148f78ac20ed7d5d993fd6bfe72540448f91bba6954af51674cb64f468556c1 not found: ID does not exist"
Jul 23 22:24:48.734447 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.734427    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:48.736508 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.736487    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dbf777bdf-zhtmj"]
Jul 23 22:24:48.744640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744622    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:48.744695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744643    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-oauth-config\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:48.744695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744654    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-config\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:48.744695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744663    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l594\" (UniqueName: \"kubernetes.io/projected/08f36af2-5036-47c1-a42a-80b7c0dfd769-kube-api-access-8l594\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:48.744695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744671    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08f36af2-5036-47c1-a42a-80b7c0dfd769-console-serving-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:48.744695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:48.744680    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08f36af2-5036-47c1-a42a-80b7c0dfd769-oauth-serving-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:24:50.112709 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:24:50.112674    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f36af2-5036-47c1-a42a-80b7c0dfd769" path="/var/lib/kubelet/pods/08f36af2-5036-47c1-a42a-80b7c0dfd769/volumes"
Jul 23 22:25:05.418511 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:05.418469    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:05.478290 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:05.478260    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:05.788731 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:05.788703    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.549382    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.549954    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="prometheus" containerID="cri-o://b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23" gracePeriod=600
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.550242    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy" containerID="cri-o://68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef" gracePeriod=600
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.550373    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-web" containerID="cri-o://05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148" gracePeriod=600
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.550474    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a" gracePeriod=600
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.550509    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="thanos-sidecar" containerID="cri-o://5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22" gracePeriod=600
Jul 23 22:25:23.550919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.550535    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="config-reloader" containerID="cri-o://fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9" gracePeriod=600
Jul 23 22:25:23.804816 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.804757    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:23.840540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840504    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a" exitCode=0
Jul 23 22:25:23.840540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840530    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef" exitCode=0
Jul 23 22:25:23.840540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840539    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148" exitCode=0
Jul 23 22:25:23.840540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840547    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22" exitCode=0
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840555    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9" exitCode=0
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840561    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af60bac-6d3f-4425-a992-8677bafa2636" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23" exitCode=0
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840586    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"}
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"}
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840760    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"}
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840793    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"}
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840800    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.840834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"}
Jul 23 22:25:23.841243 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"}
Jul 23 22:25:23.841243 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.840876    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6af60bac-6d3f-4425-a992-8677bafa2636","Type":"ContainerDied","Data":"0fd9cb8716064bc56e13263d2e7be8a5f4d1ea31b50c888f00a31f2e069c03d9"}
Jul 23 22:25:23.851959 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.849974    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.854054 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854010    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854178 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854097    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854178 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854133    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854178 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854161    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854184    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854216    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854243    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854285    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854309    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854337    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854372    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854400    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854434    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854466    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-922zb\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854521    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854562    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854604    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.854640 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854629    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.855135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854647    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.855135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854662    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls\") pod \"6af60bac-6d3f-4425-a992-8677bafa2636\" (UID: \"6af60bac-6d3f-4425-a992-8677bafa2636\") "
Jul 23 22:25:23.855135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.854865    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.855735 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.855703    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.855835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.855711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.855835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.855747    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:25:23.861962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.861137    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.861962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.861212    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out" (OuterVolumeSpecName: "config-out") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:25:23.861962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.861887    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.861962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.861924    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.863010 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.862596    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.863167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863058    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:25:23.863258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863186    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.863434 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863401    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config" (OuterVolumeSpecName: "config") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.863434 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863423    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.863602 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863468    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.863602 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.863545    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb" (OuterVolumeSpecName: "kube-api-access-922zb") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "kube-api-access-922zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:25:23.864606 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.864579    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.864746 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.864670    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:25:23.864802 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.864780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.865147 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.865114    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.869863 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.867589    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.877977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.877941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config" (OuterVolumeSpecName: "web-config") pod "6af60bac-6d3f-4425-a992-8677bafa2636" (UID: "6af60bac-6d3f-4425-a992-8677bafa2636"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:25:23.880758 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.880637    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.893532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.893501    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.901826 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.901802    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.910421 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.910403    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.916940 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.916881    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.917205 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.917180    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.917282 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917213    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.917282 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917233    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.917528 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.917503    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.917615 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917535    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.917615 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917553    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.917773 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.917753    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.917835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917783    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.917835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.917806    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.918075 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.918059    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.918122 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918080    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.918122 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918094    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.918319 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.918302    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.918363 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918327    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.918363 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918343    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.918607 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.918590    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.918650 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918612    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.918688 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918626    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.918882 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:25:23.918868    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.918951 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918884    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.918951 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.918917    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.919131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919114    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.919177 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919132    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.919326 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919309    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.919370 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919327    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.919577 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919559    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.919625 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919577    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.919810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919794    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.919853 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.919811    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.920041 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920024    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.920092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920041    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.920249 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920234    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.920284 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920249    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.920488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920468    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.920531 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920490    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.920726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920703    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.920779 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920727    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.920941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920925    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.920941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.920941    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.921182 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921160    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.921282 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921184    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.921429 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921408    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.921480 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921431    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.921653 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921633    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.921705 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921656    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.921849 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921830    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.921886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.921850    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.922056 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922041    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.922101 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922058    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.922224 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922207    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.922273 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922224    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.922401 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922386    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.922446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922401    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.922574 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922559    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.922611 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922574    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.922755 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922737    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.922798 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922755    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.922957 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922933    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.922957 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.922955    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.923131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923114    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.923181 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923132    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.923339 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923324    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.923339 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923339    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.923522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923506    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.923564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923522    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.923720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923702    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.923792 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923722    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.923986 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923968    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.924039 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.923987    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.924193 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924178    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.924239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924194    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.924394 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924378    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.924444 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924396    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.924594 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924579    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.924637 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924595    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.924766 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924751    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.924803 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924766    2559 scope.go:117] "RemoveContainer" containerID="d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"
Jul 23 22:25:23.924973 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924958    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a"} err="failed to get container status \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": rpc error: code = NotFound desc = could not find container \"d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a\": container with ID starting with d98d04e901d603279fe4960b10d694d823b6d4b06a4006692ee6829e8049196a not found: ID does not exist"
Jul 23 22:25:23.925024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.924974    2559 scope.go:117] "RemoveContainer" containerID="68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"
Jul 23 22:25:23.925212 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925184    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef"} err="failed to get container status \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": rpc error: code = NotFound desc = could not find container \"68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef\": container with ID starting with 68fb5ea7f3e1fc3e9729f8e14e1b1f88057e4504628b487fe34719c56ab4b3ef not found: ID does not exist"
Jul 23 22:25:23.925279 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925214    2559 scope.go:117] "RemoveContainer" containerID="05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"
Jul 23 22:25:23.925450 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925433    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148"} err="failed to get container status \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": rpc error: code = NotFound desc = could not find container \"05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148\": container with ID starting with 05ac801d5298d1bd211b73e09d2a7f3859e0fd3cbaf202ca5d4c09dc468d7148 not found: ID does not exist"
Jul 23 22:25:23.925501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925451    2559 scope.go:117] "RemoveContainer" containerID="5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"
Jul 23 22:25:23.925670 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925652    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22"} err="failed to get container status \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": rpc error: code = NotFound desc = could not find container \"5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22\": container with ID starting with 5efcc05854ce79e2f66361677ebe1f48088c51b449e00cc1f11a62e34c2f4a22 not found: ID does not exist"
Jul 23 22:25:23.925735 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925672    2559 scope.go:117] "RemoveContainer" containerID="fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"
Jul 23 22:25:23.925887 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925871    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9"} err="failed to get container status \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": rpc error: code = NotFound desc = could not find container \"fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9\": container with ID starting with fa2b96e5d357a3c9e639dd861cd28d896109460ef9c6223ff3300adcc1e5abe9 not found: ID does not exist"
Jul 23 22:25:23.925954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.925887    2559 scope.go:117] "RemoveContainer" containerID="b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"
Jul 23 22:25:23.926104 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.926088    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23"} err="failed to get container status \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": rpc error: code = NotFound desc = could not find container \"b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23\": container with ID starting with b7b15b2debbd11cd0e93b8b7ff49466b14081ebf10c258de2e0d384bee655c23 not found: ID does not exist"
Jul 23 22:25:23.926147 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.926105    2559 scope.go:117] "RemoveContainer" containerID="3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"
Jul 23 22:25:23.926301 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.926281    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b"} err="failed to get container status \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": rpc error: code = NotFound desc = could not find container \"3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b\": container with ID starting with 3e9a7c0743ad966f0f4ade59eefb769b1aa58fd3a1b100e339627af3c29c0c3b not found: ID does not exist"
Jul 23 22:25:23.955906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955868    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-config\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955923    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-metrics-client-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955934    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955947    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955957    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-db\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955965    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955974    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955983    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955991    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af60bac-6d3f-4425-a992-8677bafa2636-config-out\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.955999    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-metrics-client-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956006    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-web-config\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956014    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-tls-assets\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956021    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956029    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-grpc-tls\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956040    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956048    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956052 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956056    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-922zb\" (UniqueName: \"kubernetes.io/projected/6af60bac-6d3f-4425-a992-8677bafa2636-kube-api-access-922zb\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956516 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956064    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af60bac-6d3f-4425-a992-8677bafa2636-secret-kube-rbac-proxy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:23.956516 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:23.956074    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af60bac-6d3f-4425-a992-8677bafa2636-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:25:24.159214 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.159175    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:24.164447 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.164421    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:24.199922 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.199828    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:24.200182 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200164    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f36af2-5036-47c1-a42a-80b7c0dfd769" containerName="console"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200184    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f36af2-5036-47c1-a42a-80b7c0dfd769" containerName="console"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200195    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="prometheus"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200203    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="prometheus"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200215    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-thanos"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200223    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-thanos"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200238    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="init-config-reloader"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200246    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="init-config-reloader"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200254    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy"
Jul 23 22:25:24.200258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200262    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200287    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-web"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200297    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-web"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200306    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="config-reloader"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200315    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="config-reloader"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200323    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="thanos-sidecar"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200332    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="thanos-sidecar"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200389    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="thanos-sidecar"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200401    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-thanos"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200411    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f36af2-5036-47c1-a42a-80b7c0dfd769" containerName="console"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200420    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="config-reloader"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200428    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy-web"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200436    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="prometheus"
Jul 23 22:25:24.200668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.200444    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" containerName="kube-rbac-proxy"
Jul 23 22:25:24.204222 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.204201    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.206673 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.206653    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 22:25:24.209136 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209114    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-diqrcerm8me43\""
Jul 23 22:25:24.209524 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209507    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 22:25:24.209642 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209590    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 22:25:24.209708 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209664    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 22:25:24.209856 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209839    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 22:25:24.209978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209853    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 22:25:24.209978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209865    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 22:25:24.209978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.209971    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 22:25:24.210202 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.210187    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n92zj\""
Jul 23 22:25:24.210256 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.210234    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 22:25:24.210256 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.210240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 22:25:24.210461 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.210446    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 22:25:24.210665 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.210648    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 22:25:24.212504 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.212459    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 22:25:24.215676 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.215653    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 22:25:24.223867 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.223847    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.257969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258052    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.257701 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lzzj\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-kube-api-access-8lzzj\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.258683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.258415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.358986 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.358949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.358986 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.358989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359208 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359297 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359379 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359433 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359441    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359586 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359586 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359534    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lzzj\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-kube-api-access-8lzzj\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359586 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.359941 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.360186 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.360244 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.360188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.361479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.361449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.361583 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.359729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.362186 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.362157    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.362525 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.362495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.362934 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.362697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.362934 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.362837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.363128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.363105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.363296 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.363254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.363404 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.363381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.364115 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.363513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.364202 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.364140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a561679a-85b0-4e7b-954a-0b134bb1cfc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.364859 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.364840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.364970 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.364873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.365068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.365046    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.365233 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.365214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-config\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.365349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.365333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a561679a-85b0-4e7b-954a-0b134bb1cfc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.366053 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.366014    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.370341 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.370324    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lzzj\" (UniqueName: \"kubernetes.io/projected/a561679a-85b0-4e7b-954a-0b134bb1cfc4-kube-api-access-8lzzj\") pod \"prometheus-k8s-0\" (UID: \"a561679a-85b0-4e7b-954a-0b134bb1cfc4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.514566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.514524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:25:24.656285 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.656207    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 22:25:24.660166 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:25:24.660138    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda561679a_85b0_4e7b_954a_0b134bb1cfc4.slice/crio-52c4a05fc9697d65f2bbcc8499f6ed347ec24f24901b52c371b8e44559b8afc6 WatchSource:0}: Error finding container 52c4a05fc9697d65f2bbcc8499f6ed347ec24f24901b52c371b8e44559b8afc6: Status 404 returned error can't find the container with id 52c4a05fc9697d65f2bbcc8499f6ed347ec24f24901b52c371b8e44559b8afc6
Jul 23 22:25:24.846430 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.846391    2559 generic.go:356] "Generic (PLEG): container finished" podID="a561679a-85b0-4e7b-954a-0b134bb1cfc4" containerID="8e33c5b1bd99da9228174a83a1feb7d73cd474c634b9c3360527bef06c852a3f" exitCode=0
Jul 23 22:25:24.846617 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.846442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerDied","Data":"8e33c5b1bd99da9228174a83a1feb7d73cd474c634b9c3360527bef06c852a3f"}
Jul 23 22:25:24.846617 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:24.846541    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"52c4a05fc9697d65f2bbcc8499f6ed347ec24f24901b52c371b8e44559b8afc6"}
Jul 23 22:25:25.852881 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.852789    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"e110cdde2fbb102f52c0bf903dab9c9de5ea912e4847c7bffb44a7c09da296de"}
Jul 23 22:25:25.852881 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.852870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"4a8711db5aa9fcbeec856f3bdfc4ae5ec6705c527f4ce603e4c2119c00d6b9de"}
Jul 23 22:25:25.853320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.852927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"abf24dd0ed2138cd83d89c8160db7414a026ca33c165d30460d85f38efdbc093"}
Jul 23 22:25:25.853320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.852965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"69189ced19d63fd80f5525e8a9dbcdd120c1f01b06a4beae7e5d379da134c647"}
Jul 23 22:25:25.853320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.853003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"9f3b2ac1eda4b89bd53143d11cf5a086e8c635acb2c7dfd53e67d1c28198ba5a"}
Jul 23 22:25:25.853320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.853046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a561679a-85b0-4e7b-954a-0b134bb1cfc4","Type":"ContainerStarted","Data":"0b7ca0c109f356e3f0266027aa557352aafa015a0cc4ece1472056789a8ebe90"}
Jul 23 22:25:25.887839 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:25.887782    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.86937949 podStartE2EDuration="1.887766145s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.386655ms" isStatefulPod=false podCreationTimestamp="2026-07-23 22:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:25:25.885655336 +0000 UTC m=+156.249321739" watchObservedRunningTime="2026-07-23 22:25:25.887766145 +0000 UTC m=+156.251432527"
Jul 23 22:25:26.109815 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:26.109741    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af60bac-6d3f-4425-a992-8677bafa2636" path="/var/lib/kubelet/pods/6af60bac-6d3f-4425-a992-8677bafa2636/volumes"
Jul 23 22:25:29.515284 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:25:29.515189    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:26:24.515121 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:24.515084    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:26:24.531213 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:24.531188    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:26:25.088790 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:25.088760    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 22:26:40.333107 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.333068    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n62l8"]
Jul 23 22:26:40.334949 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.334934    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.337467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.337445    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 22:26:40.337467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.337456    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 22:26:40.338363 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.338349    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kz5fw\""
Jul 23 22:26:40.343651 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.343631    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n62l8"]
Jul 23 22:26:40.445906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.445863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-bound-sa-token\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.446096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.445925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ddh6\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-kube-api-access-6ddh6\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.546386 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.546343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-bound-sa-token\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.546386 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.546387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ddh6\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-kube-api-access-6ddh6\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.556320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.556290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-bound-sa-token\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.556427 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.556352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ddh6\" (UniqueName: \"kubernetes.io/projected/1f1c34f4-530e-45ca-bb1b-0d2f519eb246-kube-api-access-6ddh6\") pod \"cert-manager-55b4d954d5-n62l8\" (UID: \"1f1c34f4-530e-45ca-bb1b-0d2f519eb246\") " pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.657162 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.657085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n62l8"
Jul 23 22:26:40.785045 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:40.784861    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n62l8"]
Jul 23 22:26:40.787601 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:26:40.787576    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f1c34f4_530e_45ca_bb1b_0d2f519eb246.slice/crio-6d16d213e0fdaac2c05427b5f2c3c4a0d78c74952764793d342cd1944243ef29 WatchSource:0}: Error finding container 6d16d213e0fdaac2c05427b5f2c3c4a0d78c74952764793d342cd1944243ef29: Status 404 returned error can't find the container with id 6d16d213e0fdaac2c05427b5f2c3c4a0d78c74952764793d342cd1944243ef29
Jul 23 22:26:41.126973 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:41.126933    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n62l8" event={"ID":"1f1c34f4-530e-45ca-bb1b-0d2f519eb246","Type":"ContainerStarted","Data":"6d16d213e0fdaac2c05427b5f2c3c4a0d78c74952764793d342cd1944243ef29"}
Jul 23 22:26:44.141464 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:44.141377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n62l8" event={"ID":"1f1c34f4-530e-45ca-bb1b-0d2f519eb246","Type":"ContainerStarted","Data":"1610d2a6039928a5e41549bd857c3a28739c79dc719b8b16833b937905b1d397"}
Jul 23 22:26:44.166547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:44.166500    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n62l8" podStartSLOduration=1.197828447 podStartE2EDuration="4.166485315s" totalImagesPullingTime="2.968656868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:26:44.165362378 +0000 UTC m=+234.529028762" watchObservedRunningTime="2026-07-23 22:26:44.166485315 +0000 UTC m=+234.530151698"
Jul 23 22:26:47.003189 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.003153    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"]
Jul 23 22:26:47.005188 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.005171    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.008684 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.008655    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rpsd4\""
Jul 23 22:26:47.008800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.008687    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 22:26:47.009368 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.009350    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 22:26:47.023224 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.023194    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"]
Jul 23 22:26:47.106483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.106451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7hzk\" (UniqueName: \"kubernetes.io/projected/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-kube-api-access-b7hzk\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.106657 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.106516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-tmp\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.207335 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.207305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-tmp\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.207335 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.207348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7hzk\" (UniqueName: \"kubernetes.io/projected/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-kube-api-access-b7hzk\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.207711 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.207690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-tmp\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.222189 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.222159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7hzk\" (UniqueName: \"kubernetes.io/projected/0e4e3cc8-69b5-444b-84e7-b11a3953cc75-kube-api-access-b7hzk\") pod \"openshift-lws-operator-fd8ccff4c-slzcj\" (UID: \"0e4e3cc8-69b5-444b-84e7-b11a3953cc75\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.314190 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.314108    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"
Jul 23 22:26:47.445240 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:47.445213    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj"]
Jul 23 22:26:47.447721 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:26:47.447693    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e4e3cc8_69b5_444b_84e7_b11a3953cc75.slice/crio-82024ebe0b7262342096d015bef431783a6d7df2284b95999ca0c11dfa17bce1 WatchSource:0}: Error finding container 82024ebe0b7262342096d015bef431783a6d7df2284b95999ca0c11dfa17bce1: Status 404 returned error can't find the container with id 82024ebe0b7262342096d015bef431783a6d7df2284b95999ca0c11dfa17bce1
Jul 23 22:26:48.156774 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:48.156740    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj" event={"ID":"0e4e3cc8-69b5-444b-84e7-b11a3953cc75","Type":"ContainerStarted","Data":"82024ebe0b7262342096d015bef431783a6d7df2284b95999ca0c11dfa17bce1"}
Jul 23 22:26:50.165885 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:50.165844    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj" event={"ID":"0e4e3cc8-69b5-444b-84e7-b11a3953cc75","Type":"ContainerStarted","Data":"8106c45d7ae41701f1862cc892aa2e5cd30a02926ecb8c77268eb45b0500f6eb"}
Jul 23 22:26:50.185055 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:26:50.185004    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-slzcj" podStartSLOduration=1.683583729 podStartE2EDuration="4.184983723s" totalImagesPullingTime="2.501399994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:26:50.183660683 +0000 UTC m=+240.547327066" watchObservedRunningTime="2026-07-23 22:26:50.184983723 +0000 UTC m=+240.548650108"
Jul 23 22:27:09.701576 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.701542    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"]
Jul 23 22:27:09.704179 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.704164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.711384 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.711366    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 22:27:09.711668 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.711655    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 22:27:09.711721 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.711698    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 22:27:09.711981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.711964    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 22:27:09.712377 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.712362    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5vgr8\""
Jul 23 22:27:09.748131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.748110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"]
Jul 23 22:27:09.800571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.800539    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.800571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.800575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-manifests\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.800797 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.800595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-tmp\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.800797 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.800641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mkf2\" (UniqueName: \"kubernetes.io/projected/5f953a44-525d-4a52-b20c-5207460cbf28-kube-api-access-6mkf2\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.800797 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.800664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-webhook-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.901832 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.901799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mkf2\" (UniqueName: \"kubernetes.io/projected/5f953a44-525d-4a52-b20c-5207460cbf28-kube-api-access-6mkf2\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.901832 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.901836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-webhook-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.902107 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.901911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.902107 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.901929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-manifests\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.902107 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.901951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-tmp\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.902409 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.902383    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-manifests\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.904302 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.904274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f953a44-525d-4a52-b20c-5207460cbf28-tmp\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.904544 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.904523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-webhook-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.904604 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.904528    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f953a44-525d-4a52-b20c-5207460cbf28-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:09.921308 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:09.921281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mkf2\" (UniqueName: \"kubernetes.io/projected/5f953a44-525d-4a52-b20c-5207460cbf28-kube-api-access-6mkf2\") pod \"opendatahub-operator-controller-manager-fc7cc8c86-zqkh2\" (UID: \"5f953a44-525d-4a52-b20c-5207460cbf28\") " pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:10.014542 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:10.014513    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:10.158137 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:10.158107    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"]
Jul 23 22:27:10.161572 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:27:10.161544    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f953a44_525d_4a52_b20c_5207460cbf28.slice/crio-8192493f626c1f7bf3b66ab4f951e001321f08fda72d94c34a56fe76a23baf80 WatchSource:0}: Error finding container 8192493f626c1f7bf3b66ab4f951e001321f08fda72d94c34a56fe76a23baf80: Status 404 returned error can't find the container with id 8192493f626c1f7bf3b66ab4f951e001321f08fda72d94c34a56fe76a23baf80
Jul 23 22:27:10.241325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:10.241292    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2" event={"ID":"5f953a44-525d-4a52-b20c-5207460cbf28","Type":"ContainerStarted","Data":"8192493f626c1f7bf3b66ab4f951e001321f08fda72d94c34a56fe76a23baf80"}
Jul 23 22:27:14.262060 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:14.262021    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2" event={"ID":"5f953a44-525d-4a52-b20c-5207460cbf28","Type":"ContainerStarted","Data":"a26eadbcd13d86609e3c7e30c05cf2a351dc840fcaa98312a31f5de6669e9014"}
Jul 23 22:27:15.267138 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:15.267108    2559 generic.go:356] "Generic (PLEG): container finished" podID="5f953a44-525d-4a52-b20c-5207460cbf28" containerID="a26eadbcd13d86609e3c7e30c05cf2a351dc840fcaa98312a31f5de6669e9014" exitCode=0
Jul 23 22:27:15.267626 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:15.267196    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2" event={"ID":"5f953a44-525d-4a52-b20c-5207460cbf28","Type":"ContainerDied","Data":"a26eadbcd13d86609e3c7e30c05cf2a351dc840fcaa98312a31f5de6669e9014"}
Jul 23 22:27:16.273124 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:16.273089    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2" event={"ID":"5f953a44-525d-4a52-b20c-5207460cbf28","Type":"ContainerStarted","Data":"1fcea3757ea5a32e15648aa3b22a7149353bf55c6a42e351939091599c33eb15"}
Jul 23 22:27:16.273527 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:16.273204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:16.298808 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:16.298759    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2" podStartSLOduration=2.576053307 podStartE2EDuration="7.298743371s" totalImagesPullingTime="3.66443491s" totalInitContainerRuntime="1.058255154s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:27:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:27:16.297846564 +0000 UTC m=+266.661512948" watchObservedRunningTime="2026-07-23 22:27:16.298743371 +0000 UTC m=+266.662409754"
Jul 23 22:27:27.279863 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:27.279831    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fc7cc8c86-zqkh2"
Jul 23 22:27:29.619554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.619477    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"]
Jul 23 22:27:29.627614 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.627586    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.630454 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.630435    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 22:27:29.630716 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.630699    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 22:27:29.631022 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.631005    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 22:27:29.631405 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.631392    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 22:27:29.631614 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.631603    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m5n9l\""
Jul 23 22:27:29.636365 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.636347    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"]
Jul 23 22:27:29.649229 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.649201    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr7r8\" (UniqueName: \"kubernetes.io/projected/daade4cc-dddb-4fb9-a2b1-cab1e120252a-kube-api-access-nr7r8\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.649331 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.649236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.649331 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.649253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tmp\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.750526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.750487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr7r8\" (UniqueName: \"kubernetes.io/projected/daade4cc-dddb-4fb9-a2b1-cab1e120252a-kube-api-access-nr7r8\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.750526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.750531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.750731 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.750549    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tmp\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.750731 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:27:29.750649    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 23 22:27:29.750801 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:27:29.750733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs podName:daade4cc-dddb-4fb9-a2b1-cab1e120252a nodeName:}" failed. No retries permitted until 2026-07-23 22:27:30.250711667 +0000 UTC m=+280.614378028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs") pod "kube-auth-proxy-699c848bdc-6d9cc" (UID: "daade4cc-dddb-4fb9-a2b1-cab1e120252a") : secret "kube-auth-proxy-tls" not found
Jul 23 22:27:29.752836 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.752820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tmp\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:29.771296 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:29.771266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr7r8\" (UniqueName: \"kubernetes.io/projected/daade4cc-dddb-4fb9-a2b1-cab1e120252a-kube-api-access-nr7r8\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:30.254530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:30.254491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:30.257071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:30.257051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daade4cc-dddb-4fb9-a2b1-cab1e120252a-tls-certs\") pod \"kube-auth-proxy-699c848bdc-6d9cc\" (UID: \"daade4cc-dddb-4fb9-a2b1-cab1e120252a\") " pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:30.538415 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:30.538329    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"
Jul 23 22:27:30.674614 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:30.674587    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc"]
Jul 23 22:27:30.678818 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:27:30.678794    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaade4cc_dddb_4fb9_a2b1_cab1e120252a.slice/crio-592297c96670dc8f32b2749f1e03de323aeab19f908bd2a9536facb70fb102eb WatchSource:0}: Error finding container 592297c96670dc8f32b2749f1e03de323aeab19f908bd2a9536facb70fb102eb: Status 404 returned error can't find the container with id 592297c96670dc8f32b2749f1e03de323aeab19f908bd2a9536facb70fb102eb
Jul 23 22:27:31.331464 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:31.331422    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc" event={"ID":"daade4cc-dddb-4fb9-a2b1-cab1e120252a","Type":"ContainerStarted","Data":"592297c96670dc8f32b2749f1e03de323aeab19f908bd2a9536facb70fb102eb"}
Jul 23 22:27:33.341575 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:33.341543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc" event={"ID":"daade4cc-dddb-4fb9-a2b1-cab1e120252a","Type":"ContainerStarted","Data":"f20f55de47d6c39c0c5e41cecced06fad28ebd9235af63f0f598cb16afef90c0"}
Jul 23 22:27:33.360006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:33.359958    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-699c848bdc-6d9cc" podStartSLOduration=1.798007801 podStartE2EDuration="4.359942143s" totalImagesPullingTime="2.561934342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:27:33.358705295 +0000 UTC m=+283.722371670" watchObservedRunningTime="2026-07-23 22:27:33.359942143 +0000 UTC m=+283.723608525"
Jul 23 22:27:37.577666 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.577626    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"]
Jul 23 22:27:37.579728 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.579712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.583436 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.583415    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lchvr\""
Jul 23 22:27:37.587306 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.587286    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 22:27:37.615161 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615322 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvjz\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-kube-api-access-tsvjz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615322 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615204    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615322 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615326    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f4d15f-289c-4d09-bbb6-5beebd242e91-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.615485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.615460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.619621 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.619598    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"]
Jul 23 22:27:37.716753 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716712    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvjz\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-kube-api-access-tsvjz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716848    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716886    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.716995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.716995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f4d15f-289c-4d09-bbb6-5beebd242e91-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.717419 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.717249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.717419 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.717358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.717525 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.717481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.717525 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.717501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.717840 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.717715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f4d15f-289c-4d09-bbb6-5beebd242e91-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.719325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.719289    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.719613 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.719585    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.733352 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.733324    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.733771 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.733754    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvjz\" (UniqueName: \"kubernetes.io/projected/96f4d15f-289c-4d09-bbb6-5beebd242e91-kube-api-access-tsvjz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wcqz4\" (UID: \"96f4d15f-289c-4d09-bbb6-5beebd242e91\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:37.889954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:37.889869    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:38.031239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:38.031162    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"]
Jul 23 22:27:38.033288 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:27:38.033261    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f4d15f_289c_4d09_bbb6_5beebd242e91.slice/crio-c0aade5c87bb84eee082712b90d4c62d2e84e6111cb6382d9685c7006fde0967 WatchSource:0}: Error finding container c0aade5c87bb84eee082712b90d4c62d2e84e6111cb6382d9685c7006fde0967: Status 404 returned error can't find the container with id c0aade5c87bb84eee082712b90d4c62d2e84e6111cb6382d9685c7006fde0967
Jul 23 22:27:38.362693 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:38.362649    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4" event={"ID":"96f4d15f-289c-4d09-bbb6-5beebd242e91","Type":"ContainerStarted","Data":"c0aade5c87bb84eee082712b90d4c62d2e84e6111cb6382d9685c7006fde0967"}
Jul 23 22:27:39.048950 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.048827    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"]
Jul 23 22:27:39.053299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.053276    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.057106 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.057024    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rjgzs\""
Jul 23 22:27:39.057106 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.057043    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 22:27:39.057329 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.057308    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 22:27:39.057329 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.057325    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 22:27:39.069452 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.069419    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"]
Jul 23 22:27:39.129304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.129269    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bedc5a-d031-43b3-bbd5-92040a855d0f-manager-config\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.129471 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.129384    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmrgs\" (UniqueName: \"kubernetes.io/projected/38bedc5a-d031-43b3-bbd5-92040a855d0f-kube-api-access-jmrgs\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.129471 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.129450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-metrics-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.129583 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.129513    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.230375 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.230339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bedc5a-d031-43b3-bbd5-92040a855d0f-manager-config\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.230552 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.230394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmrgs\" (UniqueName: \"kubernetes.io/projected/38bedc5a-d031-43b3-bbd5-92040a855d0f-kube-api-access-jmrgs\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.230552 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.230418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-metrics-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.230552 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.230495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.231107 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.231080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bedc5a-d031-43b3-bbd5-92040a855d0f-manager-config\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.233479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.233437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-metrics-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.233608 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.233517    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bedc5a-d031-43b3-bbd5-92040a855d0f-cert\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.248554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.248520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmrgs\" (UniqueName: \"kubernetes.io/projected/38bedc5a-d031-43b3-bbd5-92040a855d0f-kube-api-access-jmrgs\") pod \"lws-controller-manager-58df94799c-x6s5t\" (UID: \"38bedc5a-d031-43b3-bbd5-92040a855d0f\") " pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.367170 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.367097    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:39.525001 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:39.524969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"]
Jul 23 22:27:39.528036 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:27:39.528004    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bedc5a_d031_43b3_bbd5_92040a855d0f.slice/crio-4df406cdef4f1bb8902bbeb0762b8f7e7dd119cee65bd78d3159ca83ab925081 WatchSource:0}: Error finding container 4df406cdef4f1bb8902bbeb0762b8f7e7dd119cee65bd78d3159ca83ab925081: Status 404 returned error can't find the container with id 4df406cdef4f1bb8902bbeb0762b8f7e7dd119cee65bd78d3159ca83ab925081
Jul 23 22:27:40.374490 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:40.374451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t" event={"ID":"38bedc5a-d031-43b3-bbd5-92040a855d0f","Type":"ContainerStarted","Data":"4df406cdef4f1bb8902bbeb0762b8f7e7dd119cee65bd78d3159ca83ab925081"}
Jul 23 22:27:40.888367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:40.888330    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:27:40.888448 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:40.888407    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:27:40.888448 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:40.888435    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:27:41.380090 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:41.380057    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4" event={"ID":"96f4d15f-289c-4d09-bbb6-5beebd242e91","Type":"ContainerStarted","Data":"a325474a824119a5f8b2ae370082562a8f09004eda8f8f04f74832818bcaea88"}
Jul 23 22:27:41.416387 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:41.416335    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4" podStartSLOduration=1.566230455 podStartE2EDuration="4.416317652s" totalImagesPullingTime="2.850087197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:27:41.415297515 +0000 UTC m=+291.778963900" watchObservedRunningTime="2026-07-23 22:27:41.416317652 +0000 UTC m=+291.779984034"
Jul 23 22:27:41.890595 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:41.890560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:41.894985 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:41.894964    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:42.385384 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:42.385351    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:42.386526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:42.386507    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wcqz4"
Jul 23 22:27:48.413483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:48.413442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t" event={"ID":"38bedc5a-d031-43b3-bbd5-92040a855d0f","Type":"ContainerStarted","Data":"838f82000375062db221b23595cd4bc1e416c51f9b4cd5c2ed4bf919f404bc39"}
Jul 23 22:27:48.413975 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:48.413540    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:27:48.432376 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:48.432320    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t" podStartSLOduration=0.7528551 podStartE2EDuration="9.432303868s" totalImagesPullingTime="8.679448768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:27:48.431095261 +0000 UTC m=+298.794761644" watchObservedRunningTime="2026-07-23 22:27:48.432303868 +0000 UTC m=+298.795970251"
Jul 23 22:27:49.985167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:49.985134    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 22:27:59.420476 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:27:59.420443    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58df94799c-x6s5t"
Jul 23 22:29:32.226907 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.226859    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68c458576d-wjvdc"]
Jul 23 22:29:32.230274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.230255    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.232704 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.232681    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 22:29:32.232825 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.232717    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-djwzz\""
Jul 23 22:29:32.235357 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.235335    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 22:29:32.235548 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.235536    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 22:29:32.235933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.235917    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 22:29:32.236247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.236228    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 22:29:32.241932 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.241907    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c458576d-wjvdc"]
Jul 23 22:29:32.242136 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.242119    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 22:29:32.302591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-service-ca\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302593    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-oauth-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-console-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302747    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-oauth-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302931 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98j8p\" (UniqueName: \"kubernetes.io/projected/452577b2-918a-4ed1-afef-09b8174cca62-kube-api-access-98j8p\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.302931 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.302833    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-trusted-ca-bundle\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403652 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-oauth-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403666    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98j8p\" (UniqueName: \"kubernetes.io/projected/452577b2-918a-4ed1-afef-09b8174cca62-kube-api-access-98j8p\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-trusted-ca-bundle\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-service-ca\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-oauth-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.403838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.403833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-console-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.404591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.404567    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-oauth-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.404591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.404579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-service-ca\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.404747 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.404579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-console-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.405203 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.405181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452577b2-918a-4ed1-afef-09b8174cca62-trusted-ca-bundle\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.406357 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.406327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-serving-cert\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.406434 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.406408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/452577b2-918a-4ed1-afef-09b8174cca62-console-oauth-config\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.412539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.412517    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98j8p\" (UniqueName: \"kubernetes.io/projected/452577b2-918a-4ed1-afef-09b8174cca62-kube-api-access-98j8p\") pod \"console-68c458576d-wjvdc\" (UID: \"452577b2-918a-4ed1-afef-09b8174cca62\") " pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.541336 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.541239    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:32.686325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.686248    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c458576d-wjvdc"]
Jul 23 22:29:32.688752 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:29:32.688728    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452577b2_918a_4ed1_afef_09b8174cca62.slice/crio-bbfff72cb26bcfe4fedeff3d731c4da6d1d4583b5edd4797b0c95b6aebfcd46d WatchSource:0}: Error finding container bbfff72cb26bcfe4fedeff3d731c4da6d1d4583b5edd4797b0c95b6aebfcd46d: Status 404 returned error can't find the container with id bbfff72cb26bcfe4fedeff3d731c4da6d1d4583b5edd4797b0c95b6aebfcd46d
Jul 23 22:29:32.844158 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.844072    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c458576d-wjvdc" event={"ID":"452577b2-918a-4ed1-afef-09b8174cca62","Type":"ContainerStarted","Data":"e42727b88b9c1e7fc9dc371b77de221118cccf9fbc77b9899cac72894d620011"}
Jul 23 22:29:32.844158 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.844132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c458576d-wjvdc" event={"ID":"452577b2-918a-4ed1-afef-09b8174cca62","Type":"ContainerStarted","Data":"bbfff72cb26bcfe4fedeff3d731c4da6d1d4583b5edd4797b0c95b6aebfcd46d"}
Jul 23 22:29:32.865966 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:32.865879    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68c458576d-wjvdc" podStartSLOduration=0.86586085 podStartE2EDuration="865.86085ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:29:32.865048599 +0000 UTC m=+403.228715007" watchObservedRunningTime="2026-07-23 22:29:32.86586085 +0000 UTC m=+403.229527233"
Jul 23 22:29:42.541485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:42.541453    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:42.541974 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:42.541605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:42.546695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:42.546668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:42.895075 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:42.894995    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68c458576d-wjvdc"
Jul 23 22:29:58.335788 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.335751    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"]
Jul 23 22:29:58.339125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.339106    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.342111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.342087    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7dbqr\""
Jul 23 22:29:58.350622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.350592    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"]
Jul 23 22:29:58.438968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.438922    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.438968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.438966    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rcwq\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-kube-api-access-6rcwq\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439440 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439240    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b5d6ad-9f47-479b-9d42-10782203afce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.439440 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.439281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b5d6ad-9f47-479b-9d42-10782203afce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.540652 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.540652 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rcwq\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-kube-api-access-6rcwq\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540734    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b5d6ad-9f47-479b-9d42-10782203afce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b5d6ad-9f47-479b-9d42-10782203afce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.540936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.541172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.541191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541361 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.541235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541520 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.541462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.541568 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.541553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b5d6ad-9f47-479b-9d42-10782203afce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.543600 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.543571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b5d6ad-9f47-479b-9d42-10782203afce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.543777 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.543759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b5d6ad-9f47-479b-9d42-10782203afce-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.550318 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.550251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.550504 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.550484    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rcwq\" (UniqueName: \"kubernetes.io/projected/98b5d6ad-9f47-479b-9d42-10782203afce-kube-api-access-6rcwq\") pod \"maas-default-gateway-openshift-default-56654dcf7-2265k\" (UID: \"98b5d6ad-9f47-479b-9d42-10782203afce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.650626 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.650530    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:58.791355 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.791323    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"]
Jul 23 22:29:58.794390 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:29:58.794358    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b5d6ad_9f47_479b_9d42_10782203afce.slice/crio-92b868db56b06cb4c2192e7afd9bc2a834535ca61644429e059f18169e3f0bcc WatchSource:0}: Error finding container 92b868db56b06cb4c2192e7afd9bc2a834535ca61644429e059f18169e3f0bcc: Status 404 returned error can't find the container with id 92b868db56b06cb4c2192e7afd9bc2a834535ca61644429e059f18169e3f0bcc
Jul 23 22:29:58.796431 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.796400    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:29:58.796510 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.796465    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:29:58.796510 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.796493    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:29:58.960759 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.960671    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" event={"ID":"98b5d6ad-9f47-479b-9d42-10782203afce","Type":"ContainerStarted","Data":"21b1fb74806514e8aea6f657c693932b515c9144d3118617a521ab3f96c8f205"}
Jul 23 22:29:58.960928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.960774    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" event={"ID":"98b5d6ad-9f47-479b-9d42-10782203afce","Type":"ContainerStarted","Data":"92b868db56b06cb4c2192e7afd9bc2a834535ca61644429e059f18169e3f0bcc"}
Jul 23 22:29:58.986333 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:58.986282    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" podStartSLOduration=0.986266579 podStartE2EDuration="986.266579ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:29:58.98503012 +0000 UTC m=+429.348696531" watchObservedRunningTime="2026-07-23 22:29:58.986266579 +0000 UTC m=+429.349932962"
Jul 23 22:29:59.651676 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:59.651633    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:59.656329 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:59.656306    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:59.965202 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:59.965122    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:29:59.966055 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:29:59.966035    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:30:12.417064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.416964    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:30:12.420148 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.420132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.422558 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.422530    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 22:30:12.422677 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.422601    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 22:30:12.423346 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.423330    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 22:30:12.423401 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.423337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hkrpr\""
Jul 23 22:30:12.428448 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.428421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:30:12.465635 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.465601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/340a1ed3-110e-4a6d-9b3f-198076a572f8-config-file\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.465635 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.465642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr9cd\" (UniqueName: \"kubernetes.io/projected/340a1ed3-110e-4a6d-9b3f-198076a572f8-kube-api-access-gr9cd\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.468562 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.468533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:30:12.566458 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.566426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/340a1ed3-110e-4a6d-9b3f-198076a572f8-config-file\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.566458 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.566472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr9cd\" (UniqueName: \"kubernetes.io/projected/340a1ed3-110e-4a6d-9b3f-198076a572f8-kube-api-access-gr9cd\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.567130 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.567106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/340a1ed3-110e-4a6d-9b3f-198076a572f8-config-file\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.576751 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.576728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr9cd\" (UniqueName: \"kubernetes.io/projected/340a1ed3-110e-4a6d-9b3f-198076a572f8-kube-api-access-gr9cd\") pod \"limitador-limitador-54d6d988d5-qmhsn\" (UID: \"340a1ed3-110e-4a6d-9b3f-198076a572f8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.732815 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.732772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:12.868949 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.868921    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:30:12.871176 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:30:12.871148    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340a1ed3_110e_4a6d_9b3f_198076a572f8.slice/crio-6ea2b8756f99cebb73edd3c7490f0e87f48f03371a20c9754e75ba23e86e22a4 WatchSource:0}: Error finding container 6ea2b8756f99cebb73edd3c7490f0e87f48f03371a20c9754e75ba23e86e22a4: Status 404 returned error can't find the container with id 6ea2b8756f99cebb73edd3c7490f0e87f48f03371a20c9754e75ba23e86e22a4
Jul 23 22:30:12.873191 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:12.873167    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 22:30:13.021829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:13.021742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn" event={"ID":"340a1ed3-110e-4a6d-9b3f-198076a572f8","Type":"ContainerStarted","Data":"6ea2b8756f99cebb73edd3c7490f0e87f48f03371a20c9754e75ba23e86e22a4"}
Jul 23 22:30:16.042696 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:16.042661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn" event={"ID":"340a1ed3-110e-4a6d-9b3f-198076a572f8","Type":"ContainerStarted","Data":"e6c4657f115c83a6bb08ed52ce06d2ecfe17d94d8cefac96c5f198e7e69c8a47"}
Jul 23 22:30:16.043125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:16.042782    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:30:16.059419 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:16.059366    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn" podStartSLOduration=1.630634475 podStartE2EDuration="4.05934982s" totalImagesPullingTime="2.428715345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:30:16.058219228 +0000 UTC m=+446.421885612" watchObservedRunningTime="2026-07-23 22:30:16.05934982 +0000 UTC m=+446.423016202"
Jul 23 22:30:27.049119 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:30:27.049089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"
Jul 23 22:31:30.195654 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:30.195617    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:31:37.401203 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.401168    2559 generic.go:356] "Generic (PLEG): container finished" podID="98b5d6ad-9f47-479b-9d42-10782203afce" containerID="21b1fb74806514e8aea6f657c693932b515c9144d3118617a521ab3f96c8f205" exitCode=0
Jul 23 22:31:37.401606 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.401215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" event={"ID":"98b5d6ad-9f47-479b-9d42-10782203afce","Type":"ContainerDied","Data":"21b1fb74806514e8aea6f657c693932b515c9144d3118617a521ab3f96c8f205"}
Jul 23 22:31:37.401647 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.401626    2559 scope.go:117] "RemoveContainer" containerID="21b1fb74806514e8aea6f657c693932b515c9144d3118617a521ab3f96c8f205"
Jul 23 22:31:37.402627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.402599    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:31:37.402690 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.402671    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:31:37.402726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:37.402706    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:31:38.411050 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:38.411008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" event={"ID":"98b5d6ad-9f47-479b-9d42-10782203afce","Type":"ContainerStarted","Data":"0e68c9d6b67897d9af97acf0f139679309bb6b77a7ac28988cd7dc125aac7629"}
Jul 23 22:31:38.651604 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:38.651572    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:31:38.651843 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:38.651820    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2265k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Jul 23 22:31:38.651936 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:38.651875    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" podUID="98b5d6ad-9f47-479b-9d42-10782203afce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Jul 23 22:31:39.651810 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:39.651773    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2265k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Jul 23 22:31:39.652248 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:39.651837    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" podUID="98b5d6ad-9f47-479b-9d42-10782203afce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Jul 23 22:31:40.651554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:40.651470    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2265k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Jul 23 22:31:40.651554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:40.651536    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" podUID="98b5d6ad-9f47-479b-9d42-10782203afce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Jul 23 22:31:41.651832 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:41.651794    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2265k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Jul 23 22:31:41.652257 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:41.651850    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k" podUID="98b5d6ad-9f47-479b-9d42-10782203afce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Jul 23 22:31:42.656024 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:42.655993    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:31:42.656427 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:42.656284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:31:42.657061 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:42.657036    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2265k"
Jul 23 22:31:54.103027 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:31:54.102992    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:32:18.008646 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:32:18.008607    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:32:29.203916 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:32:29.203863    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:32:37.213739 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:32:37.213706    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:32:55.313773 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:32:55.313736    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:33:00.690268 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.690216    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"]
Jul 23 22:33:00.692354 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.692339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.696454 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.696423    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 22:33:00.696595 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.696483    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dxxl7\""
Jul 23 22:33:00.696595 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.696536    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 22:33:00.696595 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.696424    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 22:33:00.705121 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.705095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"]
Jul 23 22:33:00.785761 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.785761 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.786070 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.786070 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.786070 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/782484c3-e25b-472d-836a-55f797fc5805-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.786070 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.785920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n7rk\" (UniqueName: \"kubernetes.io/projected/782484c3-e25b-472d-836a-55f797fc5805-kube-api-access-5n7rk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886487 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886449    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886487 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/782484c3-e25b-472d-836a-55f797fc5805-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n7rk\" (UniqueName: \"kubernetes.io/projected/782484c3-e25b-472d-836a-55f797fc5805-kube-api-access-5n7rk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886945 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.886945 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.886918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.887056 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.887033    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.887091 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.887074    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.889004 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.888980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/782484c3-e25b-472d-836a-55f797fc5805-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.889201 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.889183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/782484c3-e25b-472d-836a-55f797fc5805-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:00.896849 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:00.896823    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n7rk\" (UniqueName: \"kubernetes.io/projected/782484c3-e25b-472d-836a-55f797fc5805-kube-api-access-5n7rk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-524rx\" (UID: \"782484c3-e25b-472d-836a-55f797fc5805\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:01.001698 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:01.001657    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:01.134268 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:01.134244    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"]
Jul 23 22:33:01.136487 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:33:01.136449    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod782484c3_e25b_472d_836a_55f797fc5805.slice/crio-2f33ca257b399b5b2f2ebf5776fd4f803df6985533b464add932f325c2ee8af0 WatchSource:0}: Error finding container 2f33ca257b399b5b2f2ebf5776fd4f803df6985533b464add932f325c2ee8af0: Status 404 returned error can't find the container with id 2f33ca257b399b5b2f2ebf5776fd4f803df6985533b464add932f325c2ee8af0
Jul 23 22:33:01.774590 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:01.774553    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx" event={"ID":"782484c3-e25b-472d-836a-55f797fc5805","Type":"ContainerStarted","Data":"2f33ca257b399b5b2f2ebf5776fd4f803df6985533b464add932f325c2ee8af0"}
Jul 23 22:33:02.781133 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:02.781099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx" event={"ID":"782484c3-e25b-472d-836a-55f797fc5805","Type":"ContainerStarted","Data":"41be7dbfacc332199860c9793217bb1df2d42d04a69d40976f614b99aeb5cb85"}
Jul 23 22:33:02.801064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:02.801007    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx" podStartSLOduration=1.796101976 podStartE2EDuration="2.80099209s" totalImagesPullingTime="1.004890114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:33:02.799588074 +0000 UTC m=+613.163254468" watchObservedRunningTime="2026-07-23 22:33:02.80099209 +0000 UTC m=+613.164658473"
Jul 23 22:33:10.110587 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:10.110559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:33:11.002127 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:11.002092    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:11.002360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:11.002342    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:11.014919 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:11.014866    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:33:11.833234 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:33:11.833193    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-524rx"
Jul 23 22:34:17.911845 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:34:17.911806    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:34:27.898088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:34:27.898048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:34:38.605975 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:34:38.605940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:36:24.206809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:36:24.206726    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:36:33.519809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:36:33.519760    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:02.303226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:02.303187    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:13.009817 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:13.009777    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:21.110348 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:21.110307    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:30.807778 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:30.807739    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:39.516180 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:39.516141    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:37:48.819784 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:37:48.819748    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:38:00.016461 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:38:00.016424    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:38:09.510128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:38:09.510090    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:38:32.219978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:38:32.219942    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:38:40.926526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:38:40.926492    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:39:05.610722 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:39:05.610680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:39:15.804598 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:39:15.804508    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:39:32.608921 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:39:32.608859    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:39:42.412050 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:39:42.412012    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:39:53.105060 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:39:53.105025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:40:02.696366 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:40:02.696328    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:40:11.294218 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:40:11.294182    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:00.000821 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:00.000780    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:09.089967 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:09.089932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:22.201912 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:22.201809    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:34.209456 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:34.209418    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:44.295088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:44.295052    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:42:55.896849 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:42:55.896810    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:43:04.398888 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:43:04.398852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:43:15.089269 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:43:15.089226    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:43:27.108631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:43:27.108593    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:43:34.999532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:43:34.999495    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:43:45.397721 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:43:45.397632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:44:10.206346 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:10.206307    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:44:17.397220 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:17.397186    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:44:27.701568 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.701531    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:44:27.704840 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.704823    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.707120 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.707099    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6b32bd24-kserve-self-signed-certs\""
Jul 23 22:44:27.723835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.723808    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:44:27.816356 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.816528 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.816528 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqn9c\" (UniqueName: \"kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.816528 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.816635 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.816635 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.816583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.917857 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.917821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.917857 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.917856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqn9c\" (UniqueName: \"kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918146 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.917884    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918146 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918146 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918354 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918406 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.918615 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.918587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.920362 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.920343    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.920646 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.920629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:27.925549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:27.925522    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqn9c\" (UniqueName: \"kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c\") pod \"e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") " pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:28.014525 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:28.014490    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:28.145123 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:28.145097    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:44:28.147119 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:44:28.147092    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05575ae0_0dfd_4039_9e37_bc6255972fef.slice/crio-169a4a458c1d6c2f7e17e03b447c9480d397589ab62b9d6b6cc0f756fc507f3f WatchSource:0}: Error finding container 169a4a458c1d6c2f7e17e03b447c9480d397589ab62b9d6b6cc0f756fc507f3f: Status 404 returned error can't find the container with id 169a4a458c1d6c2f7e17e03b447c9480d397589ab62b9d6b6cc0f756fc507f3f
Jul 23 22:44:28.852982 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:28.852944    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" event={"ID":"05575ae0-0dfd-4039-9e37-bc6255972fef","Type":"ContainerStarted","Data":"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"}
Jul 23 22:44:28.853476 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:28.852993    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" event={"ID":"05575ae0-0dfd-4039-9e37-bc6255972fef","Type":"ContainerStarted","Data":"169a4a458c1d6c2f7e17e03b447c9480d397589ab62b9d6b6cc0f756fc507f3f"}
Jul 23 22:44:28.871027 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:28.870982    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" podStartSLOduration=1.8709648859999999 podStartE2EDuration="1.870964886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:44:28.869265821 +0000 UTC m=+1299.232932224" watchObservedRunningTime="2026-07-23 22:44:28.870964886 +0000 UTC m=+1299.234631270"
Jul 23 22:44:30.599540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:30.599503    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:44:38.014719 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:38.014685    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:38.015159 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:38.014743    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:38.027441 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:38.027411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:38.908479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:38.908448    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:44:43.181078 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.181044    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:44:43.184698 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.184681    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.187030 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.187011    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6b32bd24-kserve-self-signed-certs\""
Jul 23 22:44:43.193383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.193361    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:44:43.260957 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.260919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.261145 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.260980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.261145 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.261002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.261145 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.261018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpc6g\" (UniqueName: \"kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.261145 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.261114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.261330 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.261181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361759 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361725    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpc6g\" (UniqueName: \"kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.361977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.361887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.362275 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.362251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.362327 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.362303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.362327 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.362314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.364301 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.364279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.364565 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.364548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.370090 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.370068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpc6g\" (UniqueName: \"kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g\") pod \"e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") " pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.494262 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.494231    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:43.622537 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.622504    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:44:43.625998 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:44:43.625971    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18f55129_4af1_48f5_8d90_f13c270dd9c6.slice/crio-acdff3e12054e748d41ff5709be4c63d3c92765c054311545e6bf2ef42dd139f WatchSource:0}: Error finding container acdff3e12054e748d41ff5709be4c63d3c92765c054311545e6bf2ef42dd139f: Status 404 returned error can't find the container with id acdff3e12054e748d41ff5709be4c63d3c92765c054311545e6bf2ef42dd139f
Jul 23 22:44:43.921676 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.921562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" event={"ID":"18f55129-4af1-48f5-8d90-f13c270dd9c6","Type":"ContainerStarted","Data":"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"}
Jul 23 22:44:43.921676 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.921633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" event={"ID":"18f55129-4af1-48f5-8d90-f13c270dd9c6","Type":"ContainerStarted","Data":"acdff3e12054e748d41ff5709be4c63d3c92765c054311545e6bf2ef42dd139f"}
Jul 23 22:44:43.939335 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:43.939289    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" podStartSLOduration=0.939271517 podStartE2EDuration="939.271517ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:44:43.938655053 +0000 UTC m=+1314.302321429" watchObservedRunningTime="2026-07-23 22:44:43.939271517 +0000 UTC m=+1314.302937948"
Jul 23 22:44:48.301703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:48.301672    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:44:53.494935 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:53.494887    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:53.495384 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:53.494956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:53.507340 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:53.507314    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:44:53.979032 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:44:53.979007    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:45:13.174626 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:13.174553    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:45:14.088286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:14.088253    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:45:16.903048 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:16.903008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:45:28.195455 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.195394    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" podUID="05575ae0-0dfd-4039-9e37-bc6255972fef" containerName="main" containerID="cri-o://d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e" gracePeriod=45
Jul 23 22:45:28.447400 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.447332    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:45:28.585278 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585236    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585278 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585286    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqn9c\" (UniqueName: \"kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585485    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585495    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache" (OuterVolumeSpecName: "model-cache") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:28.585695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585575    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585596    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm\") pod \"05575ae0-0dfd-4039-9e37-bc6255972fef\" (UID: \"05575ae0-0dfd-4039-9e37-bc6255972fef\") "
Jul 23 22:45:28.585695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585636    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home" (OuterVolumeSpecName: "home") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:28.585695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585661    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:28.585875 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585850    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:28.585875 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585877    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:28.586016 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.585887    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:28.587796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.587770    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm" (OuterVolumeSpecName: "dshm") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:28.587796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.587784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:45:28.587974 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.587831    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c" (OuterVolumeSpecName: "kube-api-access-pqn9c") pod "05575ae0-0dfd-4039-9e37-bc6255972fef" (UID: "05575ae0-0dfd-4039-9e37-bc6255972fef"). InnerVolumeSpecName "kube-api-access-pqn9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:45:28.686654 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.686603    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05575ae0-0dfd-4039-9e37-bc6255972fef-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:28.686654 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.686649    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05575ae0-0dfd-4039-9e37-bc6255972fef-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:28.686654 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:28.686658    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqn9c\" (UniqueName: \"kubernetes.io/projected/05575ae0-0dfd-4039-9e37-bc6255972fef-kube-api-access-pqn9c\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.109105 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.109060    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" podUID="18f55129-4af1-48f5-8d90-f13c270dd9c6" containerName="main" containerID="cri-o://d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6" gracePeriod=45
Jul 23 22:45:29.127860 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.127830    2559 generic.go:356] "Generic (PLEG): container finished" podID="05575ae0-0dfd-4039-9e37-bc6255972fef" containerID="d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e" exitCode=0
Jul 23 22:45:29.128023 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.127870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" event={"ID":"05575ae0-0dfd-4039-9e37-bc6255972fef","Type":"ContainerDied","Data":"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"}
Jul 23 22:45:29.128023 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.127921    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz" event={"ID":"05575ae0-0dfd-4039-9e37-bc6255972fef","Type":"ContainerDied","Data":"169a4a458c1d6c2f7e17e03b447c9480d397589ab62b9d6b6cc0f756fc507f3f"}
Jul 23 22:45:29.128023 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.127873    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"
Jul 23 22:45:29.128175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.127938    2559 scope.go:117] "RemoveContainer" containerID="d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"
Jul 23 22:45:29.171275 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.171245    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:45:29.177123 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.177086    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6b32bd24-kserve-77f4c887f9-bxdbz"]
Jul 23 22:45:29.222397 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.222330    2559 scope.go:117] "RemoveContainer" containerID="d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"
Jul 23 22:45:29.222714 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:45:29.222694    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e\": container with ID starting with d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e not found: ID does not exist" containerID="d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"
Jul 23 22:45:29.222757 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.222725    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e"} err="failed to get container status \"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e\": rpc error: code = NotFound desc = could not find container \"d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e\": container with ID starting with d306164e0b5538c48455a644805889c197669f33cfbe5236e395430df875912e not found: ID does not exist"
Jul 23 22:45:29.363346 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.363286    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:45:29.496830 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.496798    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.496830 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.496834    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.497081 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.496925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.497081 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.496951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.497081 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497030    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpc6g\" (UniqueName: \"kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.497081 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497070    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir\") pod \"18f55129-4af1-48f5-8d90-f13c270dd9c6\" (UID: \"18f55129-4af1-48f5-8d90-f13c270dd9c6\") "
Jul 23 22:45:29.497341 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497168    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:29.497341 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497271    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home" (OuterVolumeSpecName: "home") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:29.497341 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497327    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:29.497475 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497385    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.497475 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497399    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.497475 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.497412    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.499320 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.499297    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g" (OuterVolumeSpecName: "kube-api-access-jpc6g") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "kube-api-access-jpc6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:45:29.499591 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.499566    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:45:29.499700 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.499589    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm" (OuterVolumeSpecName: "dshm") pod "18f55129-4af1-48f5-8d90-f13c270dd9c6" (UID: "18f55129-4af1-48f5-8d90-f13c270dd9c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:45:29.598135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.598097    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f55129-4af1-48f5-8d90-f13c270dd9c6-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.598135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.598130    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpc6g\" (UniqueName: \"kubernetes.io/projected/18f55129-4af1-48f5-8d90-f13c270dd9c6-kube-api-access-jpc6g\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.598135 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.598139    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18f55129-4af1-48f5-8d90-f13c270dd9c6-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:45:29.794842 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:29.794805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:45:30.110221 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.110143    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05575ae0-0dfd-4039-9e37-bc6255972fef" path="/var/lib/kubelet/pods/05575ae0-0dfd-4039-9e37-bc6255972fef/volumes"
Jul 23 22:45:30.133763 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.133735    2559 generic.go:356] "Generic (PLEG): container finished" podID="18f55129-4af1-48f5-8d90-f13c270dd9c6" containerID="d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6" exitCode=0
Jul 23 22:45:30.133925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.133783    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"
Jul 23 22:45:30.133925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.133829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" event={"ID":"18f55129-4af1-48f5-8d90-f13c270dd9c6","Type":"ContainerDied","Data":"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"}
Jul 23 22:45:30.133925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.133875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq" event={"ID":"18f55129-4af1-48f5-8d90-f13c270dd9c6","Type":"ContainerDied","Data":"acdff3e12054e748d41ff5709be4c63d3c92765c054311545e6bf2ef42dd139f"}
Jul 23 22:45:30.133925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.133915    2559 scope.go:117] "RemoveContainer" containerID="d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"
Jul 23 22:45:30.142141 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.142120    2559 scope.go:117] "RemoveContainer" containerID="d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"
Jul 23 22:45:30.142417 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:45:30.142397    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6\": container with ID starting with d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6 not found: ID does not exist" containerID="d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"
Jul 23 22:45:30.142465 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.142428    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6"} err="failed to get container status \"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6\": rpc error: code = NotFound desc = could not find container \"d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6\": container with ID starting with d6c5438c6ee367253186f441057395d6c753890f068b3b9c8042ee0bd18e22c6 not found: ID does not exist"
Jul 23 22:45:30.150131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.150109    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:45:30.154168 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:30.154146    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6b32bd24-kserve-69979677cf-h6xfq"]
Jul 23 22:45:32.111041 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:32.111009    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f55129-4af1-48f5-8d90-f13c270dd9c6" path="/var/lib/kubelet/pods/18f55129-4af1-48f5-8d90-f13c270dd9c6/volumes"
Jul 23 22:45:35.994992 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:35.994945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:45:45.116658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:45.116620    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:45:52.699639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:45:52.699600    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:01.893536 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:01.893501    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:10.504467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:10.504429    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:20.403451 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:20.403415    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:29.006236 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:29.006199    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:38.303791 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:38.303759    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:46:52.409617 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:46:52.409579    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:01.409042 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:01.409003    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:10.395642 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:10.395597    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:22.101117 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:22.101078    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:34.001036 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:34.001003    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:43.505643 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:43.505610    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:47:55.396940 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:47:55.396885    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:06.200275 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:06.200238    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:14.603860 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:14.603771    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:21.698638 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:21.698606    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:35.101855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:35.101815    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:44.207575 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:44.207539    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:53.693671 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:53.693634    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:48:55.851653 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.851618    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852186    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05575ae0-0dfd-4039-9e37-bc6255972fef" containerName="main"
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852205    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="05575ae0-0dfd-4039-9e37-bc6255972fef" containerName="main"
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852227    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f55129-4af1-48f5-8d90-f13c270dd9c6" containerName="main"
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852234    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f55129-4af1-48f5-8d90-f13c270dd9c6" containerName="main"
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852308    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f55129-4af1-48f5-8d90-f13c270dd9c6" containerName="main"
Jul 23 22:48:55.852392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.852321    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="05575ae0-0dfd-4039-9e37-bc6255972fef" containerName="main"
Jul 23 22:48:55.855276 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.855254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.857767 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.857750    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f205eed2-openshift-default-dockercfg-nxmnm\""
Jul 23 22:48:55.875365 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.875334    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:48:55.967693 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.967693 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p845p\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.967980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.967980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.967980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967935    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.967980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967966    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.968152 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.967990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.968152 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.968021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:55.968152 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:55.968038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p845p\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068712    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.068787 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069194 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.068789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069251 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.069204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069251 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.069226    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069343 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.069317    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069405 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.069371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.069568 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.069549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.071286 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.071270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.071507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.071489    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.078424 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.078393    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p845p\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.078838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.078815    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token\") pod \"e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") " pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.165124 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.165018    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:56.302758 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.302687    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:48:56.306216 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:48:56.306187    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e376806_c079_4150_be8a_b6715a831dbd.slice/crio-f6d147db46394f5696f5b57e91f38d616494a3a278ed50d593ba39d6de3ca8f4 WatchSource:0}: Error finding container f6d147db46394f5696f5b57e91f38d616494a3a278ed50d593ba39d6de3ca8f4: Status 404 returned error can't find the container with id f6d147db46394f5696f5b57e91f38d616494a3a278ed50d593ba39d6de3ca8f4
Jul 23 22:48:56.308644 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.308610    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:48:56.308744 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.308677    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:48:56.308744 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:56.308706    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:48:57.047205 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:57.047167    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerStarted","Data":"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"}
Jul 23 22:48:57.047623 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:57.047221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerStarted","Data":"f6d147db46394f5696f5b57e91f38d616494a3a278ed50d593ba39d6de3ca8f4"}
Jul 23 22:48:57.069829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:57.069777    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podStartSLOduration=2.069731103 podStartE2EDuration="2.069731103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:48:57.069206709 +0000 UTC m=+1567.432873119" watchObservedRunningTime="2026-07-23 22:48:57.069731103 +0000 UTC m=+1567.433397485"
Jul 23 22:48:57.165868 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:57.165834    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:57.170515 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:57.170491    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:58.052707 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:58.052668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:48:58.053696 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:48:58.053676    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:04.603639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:04.603606    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:49:13.054328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.054293    2559 patch_prober.go:28] interesting pod/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 23 22:49:13.054728 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.054361    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 23 22:49:13.125247 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.125213    2559 generic.go:356] "Generic (PLEG): container finished" podID="3e376806-c079-4150-be8a-b6715a831dbd" containerID="a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188" exitCode=0
Jul 23 22:49:13.125413 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.125288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerDied","Data":"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"}
Jul 23 22:49:13.125712 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.125699    2559 scope.go:117] "RemoveContainer" containerID="a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"
Jul 23 22:49:13.126717 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.126690    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:13.126786 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.126760    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:13.126829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:13.126793    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:14.131937 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:14.131873    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerStarted","Data":"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"}
Jul 23 22:49:14.165843 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:14.165810    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:14.166033 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:14.166006    2559 patch_prober.go:28] interesting pod/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 23 22:49:14.166080 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:14.166050    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 23 22:49:15.166222 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:15.166179    2559 patch_prober.go:28] interesting pod/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 23 22:49:15.166648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:15.166247    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 23 22:49:16.166068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:16.166032    2559 patch_prober.go:28] interesting pod/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 23 22:49:16.166240 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:16.166099    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 23 22:49:17.165969 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:17.165933    2559 patch_prober.go:28] interesting pod/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 23 22:49:17.166351 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:17.165989    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 23 22:49:18.169211 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:18.169183    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:18.169576 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:18.169520    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:18.170186 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:18.170165    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:37.399718 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:37.399682    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:49:39.259641 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:39.259611    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:49:39.260089 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:39.259812    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy" containerID="cri-o://1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63" gracePeriod=30
Jul 23 22:49:40.342114 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.342082    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:49:40.345614 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.345594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.348119 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.348100    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c9c6cb93-gw-openshift-default-dockercfg-z2wdj\""
Jul 23 22:49:40.355909 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.355871    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:49:40.483852 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.483766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.483852 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.483807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.483918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.483990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95xb7\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.484060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.484097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484355 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.484152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484355 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.484196    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.484355 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.484220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585050 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.584998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585050 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95xb7\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585516 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585516 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585516 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585674 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585814    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585989 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585831    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.585989 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.585817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.588244 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.588214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.588383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.588363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.593300 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.593237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.593463 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.593448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95xb7\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7\") pod \"e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") " pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.656629 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.656594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:40.792914 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.792857    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:49:40.794337 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:49:40.794308    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc7504f_95dd_47f0_8337_d6484f48125e.slice/crio-a638440ffdd1fa7195a30b7fe9992550c639d86799df2caa631e5ac3dfaef4f9 WatchSource:0}: Error finding container a638440ffdd1fa7195a30b7fe9992550c639d86799df2caa631e5ac3dfaef4f9: Status 404 returned error can't find the container with id a638440ffdd1fa7195a30b7fe9992550c639d86799df2caa631e5ac3dfaef4f9
Jul 23 22:49:40.796529 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.796496    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:40.796587 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.796562    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:40.796622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:40.796590    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:41.259584 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:41.259550    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerStarted","Data":"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"}
Jul 23 22:49:41.259762 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:41.259600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerStarted","Data":"a638440ffdd1fa7195a30b7fe9992550c639d86799df2caa631e5ac3dfaef4f9"}
Jul 23 22:49:41.281316 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:41.281256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podStartSLOduration=1.281232992 podStartE2EDuration="1.281232992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:49:41.280152572 +0000 UTC m=+1611.643818955" watchObservedRunningTime="2026-07-23 22:49:41.281232992 +0000 UTC m=+1611.644899374"
Jul 23 22:49:41.657133 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:41.657029    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:41.661716 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:41.661695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:42.264378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:42.264344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:42.265290 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:42.265274    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:44.506226 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.506205    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:44.625367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625315    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625358    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625379    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625395    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625420    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625517    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p845p\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625645 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625578    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy\") pod \"3e376806-c079-4150-be8a-b6715a831dbd\" (UID: \"3e376806-c079-4150-be8a-b6715a831dbd\") "
Jul 23 22:49:44.625978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625682    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:49:44.625978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625862    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:49:44.625978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625889    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:49:44.625978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625939    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:49:44.626177 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.625982    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.626177 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.626003    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.626446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.626172    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:49:44.628057 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.628027    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:49:44.628057 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.628030    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:49:44.628191 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.628065    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:49:44.628191 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.628130    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p" (OuterVolumeSpecName: "kube-api-access-p845p") pod "3e376806-c079-4150-be8a-b6715a831dbd" (UID: "3e376806-c079-4150-be8a-b6715a831dbd"). InnerVolumeSpecName "kube-api-access-p845p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:49:44.727096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727046    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p845p\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-kube-api-access-p845p\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727088    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727098    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727106    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e376806-c079-4150-be8a-b6715a831dbd-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727096 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727114    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e376806-c079-4150-be8a-b6715a831dbd-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727389 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727122    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e376806-c079-4150-be8a-b6715a831dbd-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:44.727389 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:44.727129    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e376806-c079-4150-be8a-b6715a831dbd-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:49:45.281095 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.281061    2559 generic.go:356] "Generic (PLEG): container finished" podID="3e376806-c079-4150-be8a-b6715a831dbd" containerID="1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63" exitCode=0
Jul 23 22:49:45.281293 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.281115    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"
Jul 23 22:49:45.281293 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.281145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerDied","Data":"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"}
Jul 23 22:49:45.281293 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.281196    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l" event={"ID":"3e376806-c079-4150-be8a-b6715a831dbd","Type":"ContainerDied","Data":"f6d147db46394f5696f5b57e91f38d616494a3a278ed50d593ba39d6de3ca8f4"}
Jul 23 22:49:45.281293 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.281219    2559 scope.go:117] "RemoveContainer" containerID="1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"
Jul 23 22:49:45.295441 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.295422    2559 scope.go:117] "RemoveContainer" containerID="a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"
Jul 23 22:49:45.303238 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.303217    2559 scope.go:117] "RemoveContainer" containerID="1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"
Jul 23 22:49:45.303500 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:49:45.303479    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63\": container with ID starting with 1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63 not found: ID does not exist" containerID="1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"
Jul 23 22:49:45.303579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.303513    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63"} err="failed to get container status \"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63\": rpc error: code = NotFound desc = could not find container \"1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63\": container with ID starting with 1164d4c754ec8c14493f5e829692bbc503d6fb35793f840730e2cd725fda8f63 not found: ID does not exist"
Jul 23 22:49:45.303579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.303537    2559 scope.go:117] "RemoveContainer" containerID="a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"
Jul 23 22:49:45.303776 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:49:45.303758    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188\": container with ID starting with a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188 not found: ID does not exist" containerID="a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"
Jul 23 22:49:45.303814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.303783    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188"} err="failed to get container status \"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188\": rpc error: code = NotFound desc = could not find container \"a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188\": container with ID starting with a07df0740ed16b1c7b6742a728d1121311a3d16c4c8da51270e8888010ff0188 not found: ID does not exist"
Jul 23 22:49:45.309008 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.308987    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:49:45.313125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:45.313103    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f205eed2-openshift-default-6df9b449d9-f249l"]
Jul 23 22:49:46.110459 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:46.110426    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e376806-c079-4150-be8a-b6715a831dbd" path="/var/lib/kubelet/pods/3e376806-c079-4150-be8a-b6715a831dbd/volumes"
Jul 23 22:49:46.196936 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:46.196905    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:49:54.341393 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.341360    2559 generic.go:356] "Generic (PLEG): container finished" podID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerID="73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15" exitCode=0
Jul 23 22:49:54.341813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.341433    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerDied","Data":"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"}
Jul 23 22:49:54.341852 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.341815    2559 scope.go:117] "RemoveContainer" containerID="73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"
Jul 23 22:49:54.342988 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.342954    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:54.343066 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.343048    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:54.343118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:54.343101    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:49:55.348322 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:55.348288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerStarted","Data":"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"}
Jul 23 22:49:55.657217 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:55.657131    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:55.658319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:55.658291    2559 patch_prober.go:28] interesting pod/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Jul 23 22:49:55.658319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:55.658344    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Jul 23 22:49:56.657794 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:56.657753    2559 patch_prober.go:28] interesting pod/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Jul 23 22:49:56.658227 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:56.657818    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Jul 23 22:49:57.264325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:57.264284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:57.657075 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:57.656981    2559 patch_prober.go:28] interesting pod/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Jul 23 22:49:57.657075 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:57.657048    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Jul 23 22:49:58.657003 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:58.656969    2559 patch_prober.go:28] interesting pod/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Jul 23 22:49:58.657389 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:58.657032    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Jul 23 22:49:59.661568 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:59.661538    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:49:59.662546 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:49:59.662528    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:50:07.809205 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.809170    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"]
Jul 23 22:50:07.809585 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.809534    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:07.809585 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.809546    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:07.809657 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.809607    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:07.811396 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.811380    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:07.813696 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.813670    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 22:50:07.813838 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.813761    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 22:50:07.820020 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.819999    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"]
Jul 23 22:50:07.844349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.844311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl\" (UID: \"b0f20e8d-6233-4ac4-b358-7f640db28fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:07.944822 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.944785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl\" (UID: \"b0f20e8d-6233-4ac4-b358-7f640db28fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:07.945411 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:07.945390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl\" (UID: \"b0f20e8d-6233-4ac4-b358-7f640db28fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:08.123212 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:08.123131    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:08.260460 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:08.260421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"]
Jul 23 22:50:08.262822 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:50:08.262794    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f20e8d_6233_4ac4_b358_7f640db28fe1.slice/crio-9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419 WatchSource:0}: Error finding container 9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419: Status 404 returned error can't find the container with id 9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419
Jul 23 22:50:08.264596 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:08.264578    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 22:50:08.405549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:08.405451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl" event={"ID":"b0f20e8d-6233-4ac4-b358-7f640db28fe1","Type":"ContainerStarted","Data":"9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419"}
Jul 23 22:50:11.422545 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:11.422456    2559 generic.go:356] "Generic (PLEG): container finished" podID="b0f20e8d-6233-4ac4-b358-7f640db28fe1" containerID="5727a6bf0c437a11ef695a39570642c56642b50e2a46f446d160bd231ee2b2e3" exitCode=0
Jul 23 22:50:11.422545 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:11.422518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl" event={"ID":"b0f20e8d-6233-4ac4-b358-7f640db28fe1","Type":"ContainerDied","Data":"5727a6bf0c437a11ef695a39570642c56642b50e2a46f446d160bd231ee2b2e3"}
Jul 23 22:50:12.562809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:12.562787    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:12.587726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:12.587704    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca\") pod \"b0f20e8d-6233-4ac4-b358-7f640db28fe1\" (UID: \"b0f20e8d-6233-4ac4-b358-7f640db28fe1\") "
Jul 23 22:50:12.588076 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:12.588054    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0f20e8d-6233-4ac4-b358-7f640db28fe1" (UID: "b0f20e8d-6233-4ac4-b358-7f640db28fe1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:50:12.689326 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:12.689241    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f20e8d-6233-4ac4-b358-7f640db28fe1-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:13.432533 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:13.432496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl" event={"ID":"b0f20e8d-6233-4ac4-b358-7f640db28fe1","Type":"ContainerDied","Data":"9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419"}
Jul 23 22:50:13.432704 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:13.432546    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9904a8b9aa86abdc947e706572de0e1e0e4fee9b8a497e6ebba0a7b527078419"
Jul 23 22:50:13.432704 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:13.432508    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"
Jul 23 22:50:20.895620 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:20.895589    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:50:22.880592 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:22.880555    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"]
Jul 23 22:50:22.883229 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:22.883204    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c9c6cb93-zt4wl"]
Jul 23 22:50:23.088988 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.088957    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:50:23.089222 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.089200    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy" containerID="cri-o://2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b" gracePeriod=30
Jul 23 22:50:23.853870 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.853833    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:50:23.854194 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854179    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:23.854194 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854193    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:23.854298 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854215    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f20e8d-6233-4ac4-b358-7f640db28fe1" containerName="revoke-keys"
Jul 23 22:50:23.854298 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854221    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f20e8d-6233-4ac4-b358-7f640db28fe1" containerName="revoke-keys"
Jul 23 22:50:23.854298 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854268    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f20e8d-6233-4ac4-b358-7f640db28fe1" containerName="revoke-keys"
Jul 23 22:50:23.854298 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.854278    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e376806-c079-4150-be8a-b6715a831dbd" containerName="istio-proxy"
Jul 23 22:50:23.857352 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.857324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.859562 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.859542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ca65b203-openshift-default-dockercfg-cpgn7\""
Jul 23 22:50:23.870183 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.870157    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:50:23.995551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.995551 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995577    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995654    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6d5f\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:23.996017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:23.995802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.096860 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.096824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6d5f\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.096860 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.096862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097086 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.096934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097173 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097173 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097152    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097274 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097628 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097602    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097729 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097729 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097694    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.097833 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.098084 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.097954    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.099721 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.099698    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.099819 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.099806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.104593 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.104531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.104747 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.104727    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6d5f\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f\") pod \"e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") " pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.111814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.111789    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f20e8d-6233-4ac4-b358-7f640db28fe1" path="/var/lib/kubelet/pods/b0f20e8d-6233-4ac4-b358-7f640db28fe1/volumes"
Jul 23 22:50:24.167812 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.167784    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:24.316657 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.316631    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:50:24.318517 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:50:24.318488    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf7ac63_5e13_4386_bf9a_e3f0bdeea7fc.slice/crio-573a08f1ffdcb67c057c1308f5903887611939e0ae39d5a4ffa3e8c2f74cc2d7 WatchSource:0}: Error finding container 573a08f1ffdcb67c057c1308f5903887611939e0ae39d5a4ffa3e8c2f74cc2d7: Status 404 returned error can't find the container with id 573a08f1ffdcb67c057c1308f5903887611939e0ae39d5a4ffa3e8c2f74cc2d7
Jul 23 22:50:24.320590 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.320559    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:24.320658 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.320627    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:24.320696 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.320659    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:24.480090 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.480049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerStarted","Data":"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"}
Jul 23 22:50:24.480250 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.480111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerStarted","Data":"573a08f1ffdcb67c057c1308f5903887611939e0ae39d5a4ffa3e8c2f74cc2d7"}
Jul 23 22:50:24.500067 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:24.500012    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podStartSLOduration=1.499991954 podStartE2EDuration="1.499991954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:50:24.498361375 +0000 UTC m=+1654.862027798" watchObservedRunningTime="2026-07-23 22:50:24.499991954 +0000 UTC m=+1654.863658335"
Jul 23 22:50:25.168209 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:25.168171    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:25.173070 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:25.173039    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:25.485348 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:25.485318    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:25.486431 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:25.486413    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:28.342412 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.342383    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:50:28.442879 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.442796    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.442879 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.442838    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.442879 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.442865    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.442941    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.442963    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443105    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95xb7\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443153    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443175 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443172    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443425 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443176    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:50:28.443425 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443198    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket\") pod \"ebc7504f-95dd-47f0-8337-d6484f48125e\" (UID: \"ebc7504f-95dd-47f0-8337-d6484f48125e\") "
Jul 23 22:50:28.443425 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443296    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:50:28.443425 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443406    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:50:28.443631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443562    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebc7504f-95dd-47f0-8337-d6484f48125e-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.443631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443581    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.443631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443593    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.443631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443613    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:50:28.443887 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.443868    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:50:28.445429 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.445402    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:50:28.445564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.445540    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7" (OuterVolumeSpecName: "kube-api-access-95xb7") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "kube-api-access-95xb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:50:28.445747 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.445728    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:50:28.445800 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.445746    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ebc7504f-95dd-47f0-8337-d6484f48125e" (UID: "ebc7504f-95dd-47f0-8337-d6484f48125e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:50:28.499820 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.499782    2559 generic.go:356] "Generic (PLEG): container finished" podID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerID="2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b" exitCode=0
Jul 23 22:50:28.500021 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.499824    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerDied","Data":"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"}
Jul 23 22:50:28.500021 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.499832    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"
Jul 23 22:50:28.500021 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.499916    2559 scope.go:117] "RemoveContainer" containerID="2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"
Jul 23 22:50:28.500021 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.499882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s" event={"ID":"ebc7504f-95dd-47f0-8337-d6484f48125e","Type":"ContainerDied","Data":"a638440ffdd1fa7195a30b7fe9992550c639d86799df2caa631e5ac3dfaef4f9"}
Jul 23 22:50:28.509125 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.509110    2559 scope.go:117] "RemoveContainer" containerID="73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"
Jul 23 22:50:28.517059 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.517032    2559 scope.go:117] "RemoveContainer" containerID="2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"
Jul 23 22:50:28.517304 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:50:28.517282    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b\": container with ID starting with 2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b not found: ID does not exist" containerID="2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"
Jul 23 22:50:28.517457 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.517311    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b"} err="failed to get container status \"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b\": rpc error: code = NotFound desc = could not find container \"2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b\": container with ID starting with 2bcad24a904785bef9f8035671c18730af6a77dbe5101d8e293b81e1faa4b29b not found: ID does not exist"
Jul 23 22:50:28.517457 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.517341    2559 scope.go:117] "RemoveContainer" containerID="73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"
Jul 23 22:50:28.517578 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:50:28.517561    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15\": container with ID starting with 73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15 not found: ID does not exist" containerID="73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"
Jul 23 22:50:28.517612 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.517584    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15"} err="failed to get container status \"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15\": rpc error: code = NotFound desc = could not find container \"73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15\": container with ID starting with 73c868d729655725e22d8a5eb66f0b4830c941e6fcac87cd3fad36ee3e006d15 not found: ID does not exist"
Jul 23 22:50:28.531231 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.531204    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:50:28.534400 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.534378    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c9c6cb93-gw-openshift-default-5cdd8d54bc-5r98s"]
Jul 23 22:50:28.544986 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.544966    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95xb7\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-kube-api-access-95xb7\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.545049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.544991    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.545049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.545001    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.545049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.545009    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.545049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.545018    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:28.545049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:28.545025    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebc7504f-95dd-47f0-8337-d6484f48125e-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:30.110339 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:30.110309    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" path="/var/lib/kubelet/pods/ebc7504f-95dd-47f0-8337-d6484f48125e/volumes"
Jul 23 22:50:32.598670 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:32.598635    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:50:40.751654 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:40.751611    2559 patch_prober.go:28] interesting pod/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:45610->10.133.0.32:15021: read: connection reset by peer" start-of-body=
Jul 23 22:50:40.752111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:40.751708    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:45610->10.133.0.32:15021: read: connection reset by peer"
Jul 23 22:50:41.563559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.563527    2559 generic.go:356] "Generic (PLEG): container finished" podID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerID="855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9" exitCode=0
Jul 23 22:50:41.563735 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.563576    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerDied","Data":"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"}
Jul 23 22:50:41.564018 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.564003    2559 scope.go:117] "RemoveContainer" containerID="855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"
Jul 23 22:50:41.564981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.564953    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:41.565045 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.565018    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:41.565080 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:41.565058    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:50:42.570050 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:42.570016    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerStarted","Data":"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"}
Jul 23 22:50:43.168463 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:43.168425    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:43.168731 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:43.168706    2559 patch_prober.go:28] interesting pod/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 23 22:50:43.168818 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:43.168776    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 23 22:50:44.168556 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:44.168511    2559 patch_prober.go:28] interesting pod/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 23 22:50:44.168963 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:44.168577    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 23 22:50:45.168868 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:45.168829    2559 patch_prober.go:28] interesting pod/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 23 22:50:45.169278 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:45.168884    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 23 22:50:46.168353 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:46.168317    2559 patch_prober.go:28] interesting pod/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 23 22:50:46.168535 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:46.168378    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 23 22:50:47.171518 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:47.171484    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:47.171878 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:47.171771    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:47.172351 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:47.172337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:50:51.728397 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728362    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"]
Jul 23 22:50:51.728799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728693    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:50:51.728799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728715    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:50:51.728799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728734    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:50:51.728799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728740    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:50:51.728799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.728798    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:50:51.732968 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.732947    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:51.735670 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.735645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 22:50:51.735780 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.735645    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 22:50:51.739950 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.739926    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"]
Jul 23 22:50:51.785155 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.785114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c\" (UID: \"8168b4df-d276-4f25-a7b7-fd2819a3755e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:51.886181 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.886148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c\" (UID: \"8168b4df-d276-4f25-a7b7-fd2819a3755e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:51.886795 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:51.886771    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c\" (UID: \"8168b4df-d276-4f25-a7b7-fd2819a3755e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:52.044165 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:52.044073    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:52.176699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:52.176651    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"]
Jul 23 22:50:52.178917 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:50:52.178867    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8168b4df_d276_4f25_a7b7_fd2819a3755e.slice/crio-a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d WatchSource:0}: Error finding container a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d: Status 404 returned error can't find the container with id a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d
Jul 23 22:50:52.618883 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:52.618843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c" event={"ID":"8168b4df-d276-4f25-a7b7-fd2819a3755e","Type":"ContainerStarted","Data":"d41f8d5d7a8a52457b3a090f516a1f289260a4150e258cc241a5fea777a33391"}
Jul 23 22:50:52.619051 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:52.618936    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c" event={"ID":"8168b4df-d276-4f25-a7b7-fd2819a3755e","Type":"ContainerStarted","Data":"a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d"}
Jul 23 22:50:52.635451 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:52.635396    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c" podStartSLOduration=1.635380636 podStartE2EDuration="1.635380636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:50:52.633114352 +0000 UTC m=+1682.996780737" watchObservedRunningTime="2026-07-23 22:50:52.635380636 +0000 UTC m=+1682.999047019"
Jul 23 22:50:53.626120 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:53.626087    2559 generic.go:356] "Generic (PLEG): container finished" podID="8168b4df-d276-4f25-a7b7-fd2819a3755e" containerID="d41f8d5d7a8a52457b3a090f516a1f289260a4150e258cc241a5fea777a33391" exitCode=0
Jul 23 22:50:53.626501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:53.626127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c" event={"ID":"8168b4df-d276-4f25-a7b7-fd2819a3755e","Type":"ContainerDied","Data":"d41f8d5d7a8a52457b3a090f516a1f289260a4150e258cc241a5fea777a33391"}
Jul 23 22:50:54.757627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:54.757595    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:54.812397 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:54.812370    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca\") pod \"8168b4df-d276-4f25-a7b7-fd2819a3755e\" (UID: \"8168b4df-d276-4f25-a7b7-fd2819a3755e\") "
Jul 23 22:50:54.812709 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:54.812686    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8168b4df-d276-4f25-a7b7-fd2819a3755e" (UID: "8168b4df-d276-4f25-a7b7-fd2819a3755e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:50:54.913798 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:54.913711    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8168b4df-d276-4f25-a7b7-fd2819a3755e-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:50:55.637073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:55.637041    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"
Jul 23 22:50:55.637073 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:55.637055    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c" event={"ID":"8168b4df-d276-4f25-a7b7-fd2819a3755e","Type":"ContainerDied","Data":"a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d"}
Jul 23 22:50:55.637339 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:50:55.637099    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0cf431b59b754a87c23523b012f302f79e7ed7347b8faba5db3a1e2600d1d9d"
Jul 23 22:51:04.846510 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:04.846469    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:51:06.805319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:06.805282    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"]
Jul 23 22:51:06.807576 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:06.807554    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ca65b203-vzk2c"]
Jul 23 22:51:08.085671 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.085639    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:51:08.086068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.085854    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy" containerID="cri-o://d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787" gracePeriod=30
Jul 23 22:51:08.110813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.110776    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8168b4df-d276-4f25-a7b7-fd2819a3755e" path="/var/lib/kubelet/pods/8168b4df-d276-4f25-a7b7-fd2819a3755e/volumes"
Jul 23 22:51:08.892473 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.892442    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:08.892788 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.892777    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8168b4df-d276-4f25-a7b7-fd2819a3755e" containerName="revoke-keys"
Jul 23 22:51:08.892833 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.892790    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8168b4df-d276-4f25-a7b7-fd2819a3755e" containerName="revoke-keys"
Jul 23 22:51:08.892868 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.892846    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebc7504f-95dd-47f0-8337-d6484f48125e" containerName="istio-proxy"
Jul 23 22:51:08.892868 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.892856    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8168b4df-d276-4f25-a7b7-fd2819a3755e" containerName="revoke-keys"
Jul 23 22:51:08.895994 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.895977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:08.898358 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.898340    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-cd5c7e7d-openshift-default-dockercfg-brshp\""
Jul 23 22:51:08.909054 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:08.909031    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:09.042611 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.042796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcw79\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.043031 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.043031 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.042918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.143924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.143820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.143924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.143862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.143924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.143888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.143924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.143918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcw79\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.144526 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.145118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.145118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.144675    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.146621 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.146595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.147019 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.147001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.152269 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.152248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.152459 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.152436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcw79\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79\") pod \"e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") " pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.205427 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.205381    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:09.345507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.345482    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:09.347574 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:51:09.347547    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4897e3_951a_4a23_a330_ea4022338f19.slice/crio-0ebe624bf39b39b8a93fc3d712d5a4ae2e4bbe789347ab48e99fda5516a89b54 WatchSource:0}: Error finding container 0ebe624bf39b39b8a93fc3d712d5a4ae2e4bbe789347ab48e99fda5516a89b54: Status 404 returned error can't find the container with id 0ebe624bf39b39b8a93fc3d712d5a4ae2e4bbe789347ab48e99fda5516a89b54
Jul 23 22:51:09.350133 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.350101    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:09.350218 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.350172    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:09.350218 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.350201    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:09.705539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.705457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerStarted","Data":"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"}
Jul 23 22:51:09.705539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.705509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerStarted","Data":"0ebe624bf39b39b8a93fc3d712d5a4ae2e4bbe789347ab48e99fda5516a89b54"}
Jul 23 22:51:09.728167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:09.728117    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podStartSLOduration=1.7281022190000002 podStartE2EDuration="1.728102219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:51:09.725925427 +0000 UTC m=+1700.089591824" watchObservedRunningTime="2026-07-23 22:51:09.728102219 +0000 UTC m=+1700.091768601"
Jul 23 22:51:10.206426 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:10.206394    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:10.211540 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:10.211516    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:10.710995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:10.710962    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:10.711842 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:10.711825    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:13.343631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.343603    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:51:13.482533 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482540    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482603    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482626    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6d5f\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482832    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482872    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.482980 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.482956    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs\") pod \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\" (UID: \"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc\") "
Jul 23 22:51:13.483184 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.483154    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:51:13.483441 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.483361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:13.483441 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.483399    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.483639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.483524    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:13.483836 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.483808    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:13.484099 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.484074    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:13.484982 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.484961    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:51:13.485137 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.485121    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:13.485554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.485539    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:51:13.485624 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.485592    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f" (OuterVolumeSpecName: "kube-api-access-g6d5f") pod "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" (UID: "fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc"). InnerVolumeSpecName "kube-api-access-g6d5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:51:13.584687 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584653    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584687 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584682    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584687 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584694    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584704    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6d5f\" (UniqueName: \"kubernetes.io/projected/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-kube-api-access-g6d5f\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584714    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584722    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584731    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.584948 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.584739    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:13.728795 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.728755    2559 generic.go:356] "Generic (PLEG): container finished" podID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerID="d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787" exitCode=0
Jul 23 22:51:13.729000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.728802    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"
Jul 23 22:51:13.729000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.728842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerDied","Data":"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"}
Jul 23 22:51:13.729000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.728919    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz" event={"ID":"fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc","Type":"ContainerDied","Data":"573a08f1ffdcb67c057c1308f5903887611939e0ae39d5a4ffa3e8c2f74cc2d7"}
Jul 23 22:51:13.729000 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.728944    2559 scope.go:117] "RemoveContainer" containerID="d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"
Jul 23 22:51:13.738158 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.738138    2559 scope.go:117] "RemoveContainer" containerID="855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"
Jul 23 22:51:13.746203 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.746186    2559 scope.go:117] "RemoveContainer" containerID="d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"
Jul 23 22:51:13.746433 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:51:13.746414    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787\": container with ID starting with d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787 not found: ID does not exist" containerID="d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"
Jul 23 22:51:13.746483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.746443    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787"} err="failed to get container status \"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787\": rpc error: code = NotFound desc = could not find container \"d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787\": container with ID starting with d0345e1ef64f001643c12d7a1bc6300ef895d6aac805c6541ddbf19b50edf787 not found: ID does not exist"
Jul 23 22:51:13.746483 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.746461    2559 scope.go:117] "RemoveContainer" containerID="855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"
Jul 23 22:51:13.746700 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:51:13.746684    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9\": container with ID starting with 855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9 not found: ID does not exist" containerID="855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"
Jul 23 22:51:13.746747 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.746707    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9"} err="failed to get container status \"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9\": rpc error: code = NotFound desc = could not find container \"855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9\": container with ID starting with 855ce5f6563f17c79cfd2e4cffff6aa299d1ba91c5ecc13d0692269a2aab96e9 not found: ID does not exist"
Jul 23 22:51:13.749755 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.749737    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:51:13.753817 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:13.753793    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ca65b203-openshift-default-c795645f5-tcvsz"]
Jul 23 22:51:14.110163 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:14.110081    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" path="/var/lib/kubelet/pods/fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc/volumes"
Jul 23 22:51:16.894582 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:16.894546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:51:24.781637 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.781556    2559 generic.go:356] "Generic (PLEG): container finished" podID="6f4897e3-951a-4a23-a330-ea4022338f19" containerID="f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383" exitCode=0
Jul 23 22:51:24.781637 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.781608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerDied","Data":"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"}
Jul 23 22:51:24.782111 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.782033    2559 scope.go:117] "RemoveContainer" containerID="f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"
Jul 23 22:51:24.783153 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.783123    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:24.783222 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.783188    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:24.783304 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:24.783223    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:51:25.711794 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:25.711727    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:25.787674 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:25.787638    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerStarted","Data":"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"}
Jul 23 22:51:26.207008 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:26.206972    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:26.207401 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:26.207378    2559 patch_prober.go:28] interesting pod/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 23 22:51:26.207574 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:26.207551    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 23 22:51:27.206489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:27.206452    2559 patch_prober.go:28] interesting pod/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 23 22:51:27.207059 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:27.207027    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 23 22:51:28.206415 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:28.206364    2559 patch_prober.go:28] interesting pod/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 23 22:51:28.206642 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:28.206459    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 23 22:51:29.206647 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:29.206596    2559 patch_prober.go:28] interesting pod/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 23 22:51:29.207201 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:29.206666    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 23 22:51:30.210501 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:30.210475    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:30.210858 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:30.210730    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:30.211407 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:30.211391    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:35.975025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.974991    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"]
Jul 23 22:51:35.975392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975330    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.975392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975341    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.975392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975360    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.975392 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975365    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.975530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975418    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.975530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.975428    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf7ac63-5e13-4386-bf9a-e3f0bdeea7fc" containerName="istio-proxy"
Jul 23 22:51:35.977491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.977466    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:35.979989 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.979952    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 22:51:35.980095 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.980067    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 22:51:35.986005 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.985979    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"]
Jul 23 22:51:35.993534 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:35.993489    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn\" (UID: \"d0f39ef8-ae20-4b23-8967-6cece3ec96b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:36.094981 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.094950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn\" (UID: \"d0f39ef8-ae20-4b23-8967-6cece3ec96b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:36.095607 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.095589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn\" (UID: \"d0f39ef8-ae20-4b23-8967-6cece3ec96b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:36.289038 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.288959    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:36.418775 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.418750    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"]
Jul 23 22:51:36.421201 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:51:36.421174    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f39ef8_ae20_4b23_8967_6cece3ec96b2.slice/crio-c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471 WatchSource:0}: Error finding container c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471: Status 404 returned error can't find the container with id c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471
Jul 23 22:51:36.848189 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.848148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn" event={"ID":"d0f39ef8-ae20-4b23-8967-6cece3ec96b2","Type":"ContainerStarted","Data":"6ec4a292b17538276e2cb62d21f3b6478a4c5d331e405ae95b2698bca777031b"}
Jul 23 22:51:36.848387 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.848205    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn" event={"ID":"d0f39ef8-ae20-4b23-8967-6cece3ec96b2","Type":"ContainerStarted","Data":"c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471"}
Jul 23 22:51:36.864845 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:36.864685    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn" podStartSLOduration=1.864664235 podStartE2EDuration="1.864664235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:51:36.864029912 +0000 UTC m=+1727.227696307" watchObservedRunningTime="2026-07-23 22:51:36.864664235 +0000 UTC m=+1727.228330619"
Jul 23 22:51:37.854489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:37.854460    2559 generic.go:356] "Generic (PLEG): container finished" podID="d0f39ef8-ae20-4b23-8967-6cece3ec96b2" containerID="6ec4a292b17538276e2cb62d21f3b6478a4c5d331e405ae95b2698bca777031b" exitCode=0
Jul 23 22:51:37.854862 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:37.854514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn" event={"ID":"d0f39ef8-ae20-4b23-8967-6cece3ec96b2","Type":"ContainerDied","Data":"6ec4a292b17538276e2cb62d21f3b6478a4c5d331e405ae95b2698bca777031b"}
Jul 23 22:51:38.987339 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:38.987313    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:39.022071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.022046    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca\") pod \"d0f39ef8-ae20-4b23-8967-6cece3ec96b2\" (UID: \"d0f39ef8-ae20-4b23-8967-6cece3ec96b2\") "
Jul 23 22:51:39.022479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.022453    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0f39ef8-ae20-4b23-8967-6cece3ec96b2" (UID: "d0f39ef8-ae20-4b23-8967-6cece3ec96b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:51:39.123644 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.123558    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0f39ef8-ae20-4b23-8967-6cece3ec96b2-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:39.866646 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.866616    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"
Jul 23 22:51:39.866646 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.866631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn" event={"ID":"d0f39ef8-ae20-4b23-8967-6cece3ec96b2","Type":"ContainerDied","Data":"c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471"}
Jul 23 22:51:39.866932 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:39.866683    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c61785848638a522485a8ab034c01593534b5070df9526834921da6892d09471"
Jul 23 22:51:49.100537 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:49.100495    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:51:51.052388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:51.052341    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"]
Jul 23 22:51:51.053979 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:51.053959    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cd5c7e7d-wgnjn"]
Jul 23 22:51:51.209109 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:51.209076    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:51.209366 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:51.209339    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy" containerID="cri-o://bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b" gracePeriod=30
Jul 23 22:51:52.113947 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:52.113912    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0f39ef8-ae20-4b23-8967-6cece3ec96b2" path="/var/lib/kubelet/pods/d0f39ef8-ae20-4b23-8967-6cece3ec96b2/volumes"
Jul 23 22:51:56.455580 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.455546    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:56.590522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590486    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcw79\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590568    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590593    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590614    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590638    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.590713 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590698    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.591048 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590756    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.591048 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.590814    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy\") pod \"6f4897e3-951a-4a23-a330-ea4022338f19\" (UID: \"6f4897e3-951a-4a23-a330-ea4022338f19\") "
Jul 23 22:51:56.591151 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591041    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:56.591151 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:56.591264 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591186    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.591264 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591246    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:56.591394 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:51:56.591777 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.591707    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:56.592922 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.592883    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79" (OuterVolumeSpecName: "kube-api-access-jcw79") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "kube-api-access-jcw79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:51:56.593196 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.593177    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:51:56.593480 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.593459    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:51:56.593566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.593546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f4897e3-951a-4a23-a330-ea4022338f19" (UID: "6f4897e3-951a-4a23-a330-ea4022338f19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:51:56.692492 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692455    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692492 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692486    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcw79\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-kube-api-access-jcw79\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692492 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692500    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4897e3-951a-4a23-a330-ea4022338f19-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692512    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692523    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4897e3-951a-4a23-a330-ea4022338f19-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692534    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692547    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4897e3-951a-4a23-a330-ea4022338f19-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.692733 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.692560    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4897e3-951a-4a23-a330-ea4022338f19-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:51:56.951292 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.951200    2559 generic.go:356] "Generic (PLEG): container finished" podID="6f4897e3-951a-4a23-a330-ea4022338f19" containerID="bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b" exitCode=0
Jul 23 22:51:56.951292 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.951258    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"
Jul 23 22:51:56.951292 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.951282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerDied","Data":"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"}
Jul 23 22:51:56.951559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.951334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv" event={"ID":"6f4897e3-951a-4a23-a330-ea4022338f19","Type":"ContainerDied","Data":"0ebe624bf39b39b8a93fc3d712d5a4ae2e4bbe789347ab48e99fda5516a89b54"}
Jul 23 22:51:56.951559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.951357    2559 scope.go:117] "RemoveContainer" containerID="bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"
Jul 23 22:51:56.967571 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.967551    2559 scope.go:117] "RemoveContainer" containerID="f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"
Jul 23 22:51:56.975458 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.975439    2559 scope.go:117] "RemoveContainer" containerID="bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"
Jul 23 22:51:56.975719 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:51:56.975700    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b\": container with ID starting with bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b not found: ID does not exist" containerID="bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"
Jul 23 22:51:56.975770 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.975727    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b"} err="failed to get container status \"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b\": rpc error: code = NotFound desc = could not find container \"bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b\": container with ID starting with bc9d6163e9614038361c32b21ac077dd1d65e0421ab8e6d05360bc38736f1b6b not found: ID does not exist"
Jul 23 22:51:56.975770 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.975745    2559 scope.go:117] "RemoveContainer" containerID="f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"
Jul 23 22:51:56.976039 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:51:56.976023    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383\": container with ID starting with f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383 not found: ID does not exist" containerID="f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"
Jul 23 22:51:56.976103 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.976041    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383"} err="failed to get container status \"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383\": rpc error: code = NotFound desc = could not find container \"f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383\": container with ID starting with f07fbf10d6fb4c93f62d49d7224ee24022b7d48013d806df58335ab34e7b9383 not found: ID does not exist"
Jul 23 22:51:56.979069 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.979043    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:56.982784 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:56.982760    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-cd5c7e7d-openshift-default-56fdddfd75-4xdxv"]
Jul 23 22:51:58.109978 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:51:58.109950    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" path="/var/lib/kubelet/pods/6f4897e3-951a-4a23-a330-ea4022338f19/volumes"
Jul 23 22:53:17.502723 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:17.502686    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:53:27.496790 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:27.496745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:53:31.308538 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308502    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308820    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308831    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308845    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0f39ef8-ae20-4b23-8967-6cece3ec96b2" containerName="revoke-keys"
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308850    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0f39ef8-ae20-4b23-8967-6cece3ec96b2" containerName="revoke-keys"
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308860    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:31.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308865    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:31.309144 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308948    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:31.309144 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.308956    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0f39ef8-ae20-4b23-8967-6cece3ec96b2" containerName="revoke-keys"
Jul 23 22:53:31.311962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.311946    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.314506 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.314484    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d60afe1f-openshift-default-dockercfg-4jmjn\""
Jul 23 22:53:31.339002 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.338973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:53:31.461550 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461550 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461578    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461776 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcfx7\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461854    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.461954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.461873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562607 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562508    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562607 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562607 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562604    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcfx7\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562808    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.562925 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562881    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562965    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.562973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.563017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563326 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.563151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563326 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.563193    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.563685 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.563662    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.565090 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.565070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.565548 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.565533    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.578336 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.578314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcfx7\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.581299 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.581279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token\") pod \"e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") " pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.622325 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.622296    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:31.763741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.763702    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:53:31.766574 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:53:31.766543    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320a1c7d_7713_44af_a352_e5adf3e5c428.slice/crio-a33c762a73b5cccc0a1b79b0207bf98207c95d85c3eb6d2af9043c3f32bdd358 WatchSource:0}: Error finding container a33c762a73b5cccc0a1b79b0207bf98207c95d85c3eb6d2af9043c3f32bdd358: Status 404 returned error can't find the container with id a33c762a73b5cccc0a1b79b0207bf98207c95d85c3eb6d2af9043c3f32bdd358
Jul 23 22:53:31.768592 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.768561    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:31.768663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.768631    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:31.768663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:31.768660    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:32.406554 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:32.406505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerStarted","Data":"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"}
Jul 23 22:53:32.406930 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:32.406565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerStarted","Data":"a33c762a73b5cccc0a1b79b0207bf98207c95d85c3eb6d2af9043c3f32bdd358"}
Jul 23 22:53:32.427607 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:32.427557    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podStartSLOduration=1.427541194 podStartE2EDuration="1.427541194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:53:32.426012691 +0000 UTC m=+1842.789679147" watchObservedRunningTime="2026-07-23 22:53:32.427541194 +0000 UTC m=+1842.791207577"
Jul 23 22:53:32.623025 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:32.622987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:32.627856 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:32.627832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:33.410953 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:33.410921    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:33.411836 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:33.411820    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:37.295608 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:37.295573    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:53:45.470556 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.470521    2559 generic.go:356] "Generic (PLEG): container finished" podID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerID="4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede" exitCode=0
Jul 23 22:53:45.470991 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.470570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerDied","Data":"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"}
Jul 23 22:53:45.470991 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.470944    2559 scope.go:117] "RemoveContainer" containerID="4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"
Jul 23 22:53:45.471971 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.471943    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:45.472049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.472017    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:45.472087 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:45.472048    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:53:46.472118 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.472088    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"]
Jul 23 22:53:46.472522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.472511    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f4897e3-951a-4a23-a330-ea4022338f19" containerName="istio-proxy"
Jul 23 22:53:46.475803 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.475782    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:46.477201 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.477181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerStarted","Data":"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"}
Jul 23 22:53:46.478479 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.478463    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 22:53:46.478542 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.478512    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 22:53:46.484374 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.484350    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"]
Jul 23 22:53:46.506796 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.506747    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk\" (UID: \"9e13a5b0-897b-4ef7-b900-cb73ca367ce1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:46.608435 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.608387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk\" (UID: \"9e13a5b0-897b-4ef7-b900-cb73ca367ce1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:46.609311 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.609280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk\" (UID: \"9e13a5b0-897b-4ef7-b900-cb73ca367ce1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:46.623122 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.623088    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:46.624198 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.624166    2559 patch_prober.go:28] interesting pod/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 23 22:53:46.624338 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.624219    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 23 22:53:46.786582 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.786494    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:46.966338 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:46.966315    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"]
Jul 23 22:53:46.969064 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:53:46.969037    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e13a5b0_897b_4ef7_b900_cb73ca367ce1.slice/crio-ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426 WatchSource:0}: Error finding container ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426: Status 404 returned error can't find the container with id ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426
Jul 23 22:53:47.483739 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:47.483650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk" event={"ID":"9e13a5b0-897b-4ef7-b900-cb73ca367ce1","Type":"ContainerStarted","Data":"133040e45ae9f381bb757709aae26261ec40cb48742ab7b1e66f41631fd2842b"}
Jul 23 22:53:47.483739 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:47.483702    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk" event={"ID":"9e13a5b0-897b-4ef7-b900-cb73ca367ce1","Type":"ContainerStarted","Data":"ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426"}
Jul 23 22:53:47.499563 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:47.499498    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk" podStartSLOduration=1.499477059 podStartE2EDuration="1.499477059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:53:47.498712002 +0000 UTC m=+1857.862378385" watchObservedRunningTime="2026-07-23 22:53:47.499477059 +0000 UTC m=+1857.863143444"
Jul 23 22:53:47.622995 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:47.622954    2559 patch_prober.go:28] interesting pod/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 23 22:53:47.623193 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:47.623018    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 23 22:53:48.411172 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:48.411062    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:48.489374 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:48.489342    2559 generic.go:356] "Generic (PLEG): container finished" podID="9e13a5b0-897b-4ef7-b900-cb73ca367ce1" containerID="133040e45ae9f381bb757709aae26261ec40cb48742ab7b1e66f41631fd2842b" exitCode=0
Jul 23 22:53:48.489743 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:48.489436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk" event={"ID":"9e13a5b0-897b-4ef7-b900-cb73ca367ce1","Type":"ContainerDied","Data":"133040e45ae9f381bb757709aae26261ec40cb48742ab7b1e66f41631fd2842b"}
Jul 23 22:53:48.623746 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:48.623707    2559 patch_prober.go:28] interesting pod/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 23 22:53:48.623942 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:48.623770    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 23 22:53:49.623585 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.623554    2559 patch_prober.go:28] interesting pod/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 23 22:53:49.624068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.623607    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 23 22:53:49.670322 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.670291    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:49.744412 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.744380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca\") pod \"9e13a5b0-897b-4ef7-b900-cb73ca367ce1\" (UID: \"9e13a5b0-897b-4ef7-b900-cb73ca367ce1\") "
Jul 23 22:53:49.744741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.744719    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e13a5b0-897b-4ef7-b900-cb73ca367ce1" (UID: "9e13a5b0-897b-4ef7-b900-cb73ca367ce1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:53:49.845650 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:49.845559    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e13a5b0-897b-4ef7-b900-cb73ca367ce1-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:53:50.502488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:50.502460    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"
Jul 23 22:53:50.502488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:50.502453    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk" event={"ID":"9e13a5b0-897b-4ef7-b900-cb73ca367ce1","Type":"ContainerDied","Data":"ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426"}
Jul 23 22:53:50.502760 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:50.502550    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce44746797bd3a4dfb714d2edf64d130682ead5cec7f4d1948e2cac196d6a426"
Jul 23 22:53:50.626740 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:50.626711    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:50.627748 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:50.627728    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:53:59.401331 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:53:59.401290    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:54:01.543726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:01.543692    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"]
Jul 23 22:54:01.546162 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:01.546135    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d60afe1f-zjtfk"]
Jul 23 22:54:02.110292 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:02.110260    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e13a5b0-897b-4ef7-b900-cb73ca367ce1" path="/var/lib/kubelet/pods/9e13a5b0-897b-4ef7-b900-cb73ca367ce1/volumes"
Jul 23 22:54:08.496242 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:08.496206    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:54:08.496704 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:08.496474    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy" containerID="cri-o://da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f" gracePeriod=30
Jul 23 22:54:13.756194 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.756168    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:54:13.869976 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.869860    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.869976 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.869924    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.869976 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.869958    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.869976 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.869976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.869996    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870048    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870187    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcfx7\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7\") pod \"320a1c7d-7713-44af-a352-e5adf3e5c428\" (UID: \"320a1c7d-7713-44af-a352-e5adf3e5c428\") "
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870190    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:54:13.870315 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870213    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:54:13.870663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:54:13.870663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870466    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.870663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870601    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.870663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.870515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:54:13.871066 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.871032    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data" (OuterVolumeSpecName: "istio-data") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:54:13.872452 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.872426    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:54:13.872452 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.872431    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token" (OuterVolumeSpecName: "istio-token") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:54:13.872873 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.872856    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:54:13.872873 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.872863    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7" (OuterVolumeSpecName: "kube-api-access-dcfx7") pod "320a1c7d-7713-44af-a352-e5adf3e5c428" (UID: "320a1c7d-7713-44af-a352-e5adf3e5c428"). InnerVolumeSpecName "kube-api-access-dcfx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:54:13.971699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971666    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.971699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971695    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320a1c7d-7713-44af-a352-e5adf3e5c428-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.971699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971705    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.972006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971712    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.972006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971721    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.972006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971730    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcfx7\" (UniqueName: \"kubernetes.io/projected/320a1c7d-7713-44af-a352-e5adf3e5c428-kube-api-access-dcfx7\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:13.972006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:13.971741    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320a1c7d-7713-44af-a352-e5adf3e5c428-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:54:14.616624 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.616585    2559 generic.go:356] "Generic (PLEG): container finished" podID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerID="da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f" exitCode=0
Jul 23 22:54:14.616829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.616641    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"
Jul 23 22:54:14.616829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.616674    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerDied","Data":"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"}
Jul 23 22:54:14.616829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.616744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs" event={"ID":"320a1c7d-7713-44af-a352-e5adf3e5c428","Type":"ContainerDied","Data":"a33c762a73b5cccc0a1b79b0207bf98207c95d85c3eb6d2af9043c3f32bdd358"}
Jul 23 22:54:14.616829 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.616777    2559 scope.go:117] "RemoveContainer" containerID="da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"
Jul 23 22:54:14.625387 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.625369    2559 scope.go:117] "RemoveContainer" containerID="4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"
Jul 23 22:54:14.632623 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.632601    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:54:14.633548 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.633534    2559 scope.go:117] "RemoveContainer" containerID="da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"
Jul 23 22:54:14.633784 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:54:14.633767    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f\": container with ID starting with da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f not found: ID does not exist" containerID="da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"
Jul 23 22:54:14.633863 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.633790    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f"} err="failed to get container status \"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f\": rpc error: code = NotFound desc = could not find container \"da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f\": container with ID starting with da0e029b5efaa727bc4cee20bf5ad45e8aec0b20a0d2e6fa1e1cda29ce4f926f not found: ID does not exist"
Jul 23 22:54:14.633863 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.633804    2559 scope.go:117] "RemoveContainer" containerID="4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"
Jul 23 22:54:14.634321 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:54:14.634289    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede\": container with ID starting with 4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede not found: ID does not exist" containerID="4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"
Jul 23 22:54:14.634433 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.634324    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede"} err="failed to get container status \"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede\": rpc error: code = NotFound desc = could not find container \"4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede\": container with ID starting with 4aa7d0937845e4e2c7d628402ff0bfd13bd2a86c3da763d579c4a079a6bd4ede not found: ID does not exist"
Jul 23 22:54:14.636019 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:14.635998    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d60afe1f-openshift-default-7ffb8f8f95-l4zvs"]
Jul 23 22:54:16.110529 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:16.110496    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" path="/var/lib/kubelet/pods/320a1c7d-7713-44af-a352-e5adf3e5c428/volumes"
Jul 23 22:54:28.521333 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521293    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:54:28.521824 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521806    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e13a5b0-897b-4ef7-b900-cb73ca367ce1" containerName="revoke-keys"
Jul 23 22:54:28.521886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521829    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e13a5b0-897b-4ef7-b900-cb73ca367ce1" containerName="revoke-keys"
Jul 23 22:54:28.521886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521843    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:28.521886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521851    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:28.521886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521868    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:28.521886 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521876    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:28.522088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521979    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:28.522088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.521995    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e13a5b0-897b-4ef7-b900-cb73ca367ce1" containerName="revoke-keys"
Jul 23 22:54:28.524933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.524913    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.527179 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.527157    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b2ff72b5-openshift-default-dockercfg-w2p27\""
Jul 23 22:54:28.535430 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.535401    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:54:28.601209 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601209 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t852d\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601409 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601240    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601409 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601409 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601409 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.601539 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.601457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702027 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.701993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702027 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702266 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702266 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702095    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702266 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702266 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702266 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t852d\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702522 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702667 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702583    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702712 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.702749 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.702707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.703182 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.703161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.704660 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.704640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.704877 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.704859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.709962 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.709941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.710664 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.710645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t852d\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d\") pod \"e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") " pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.837197 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.837092    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:28.988390 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.988289    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:54:28.990697 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:54:28.990665    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2825e78a_1439_4017_b0a9_3452ab34f6b7.slice/crio-da1ae689a09abd6b8d2088a2fd55150684801696f173a75d2d82971f7d5f8331 WatchSource:0}: Error finding container da1ae689a09abd6b8d2088a2fd55150684801696f173a75d2d82971f7d5f8331: Status 404 returned error can't find the container with id da1ae689a09abd6b8d2088a2fd55150684801696f173a75d2d82971f7d5f8331
Jul 23 22:54:28.992835 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.992800    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:28.992933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.992865    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:28.992933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:28.992918    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:29.692638 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:29.692597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerStarted","Data":"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"}
Jul 23 22:54:29.693056 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:29.692684    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerStarted","Data":"da1ae689a09abd6b8d2088a2fd55150684801696f173a75d2d82971f7d5f8331"}
Jul 23 22:54:29.713845 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:29.713628    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podStartSLOduration=1.713605034 podStartE2EDuration="1.713605034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:54:29.712820666 +0000 UTC m=+1900.076487074" watchObservedRunningTime="2026-07-23 22:54:29.713605034 +0000 UTC m=+1900.077271423"
Jul 23 22:54:29.838230 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:29.838197    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:30.842360 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:30.842331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:31.701660 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:31.701629    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:31.702549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:31.702532    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:38.197731 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:38.197692    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:54:46.703267 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.703227    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 23 22:54:46.703744 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.703296    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 23 22:54:46.775308 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.775275    2559 generic.go:356] "Generic (PLEG): container finished" podID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerID="d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9" exitCode=0
Jul 23 22:54:46.775498 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.775354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerDied","Data":"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"}
Jul 23 22:54:46.775799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.775782    2559 scope.go:117] "RemoveContainer" containerID="d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"
Jul 23 22:54:46.776883 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.776853    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:46.776974 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.776956    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:46.777019 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:46.776999    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:54:47.499813 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.499767    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:54:47.500641 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.500619    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="320a1c7d-7713-44af-a352-e5adf3e5c428" containerName="istio-proxy"
Jul 23 22:54:47.504128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.503675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.509511 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.509482    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b2ff72b5\"/\"e2e-lifecycle-model-b2ff72b5-kserve-self-signed-certs\""
Jul 23 22:54:47.509881 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.509861    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b2ff72b5\"/\"default-dockercfg-8clvd\""
Jul 23 22:54:47.510622 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.510600    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b2ff72b5\"/\"kube-root-ca.crt\""
Jul 23 22:54:47.511285 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.511262    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b2ff72b5\"/\"openshift-service-ca.crt\""
Jul 23 22:54:47.519628 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.519602    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:54:47.588328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.588328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.588328 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.588627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flk64\" (UniqueName: \"kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.588627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.588627 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.588480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690044 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flk64\" (UniqueName: \"kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690044 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690043    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690321 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690321 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690321 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690170    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690321 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690383    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.690564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.690496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.692566 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.692538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.692799 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.692777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.699429 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.699381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flk64\" (UniqueName: \"kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64\") pod \"e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") " pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.782078 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.781984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerStarted","Data":"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"}
Jul 23 22:54:47.827884 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.827839    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:47.837927 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.837876    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:47.838148 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.838127    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 23 22:54:47.838207 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.838175    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 23 22:54:47.986691 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:47.986665    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:54:47.988742 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:54:47.988713    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c54744d_4b39_4e05_b720_9ccf21cdbd97.slice/crio-999d5fdb16f42d73caa47d3c9dc07eac15527d0f83028e9adf3d9e0eb421b14a WatchSource:0}: Error finding container 999d5fdb16f42d73caa47d3c9dc07eac15527d0f83028e9adf3d9e0eb421b14a: Status 404 returned error can't find the container with id 999d5fdb16f42d73caa47d3c9dc07eac15527d0f83028e9adf3d9e0eb421b14a
Jul 23 22:54:48.788745 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:48.788707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" event={"ID":"4c54744d-4b39-4e05-b720-9ccf21cdbd97","Type":"ContainerStarted","Data":"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"}
Jul 23 22:54:48.789199 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:48.788759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" event={"ID":"4c54744d-4b39-4e05-b720-9ccf21cdbd97","Type":"ContainerStarted","Data":"999d5fdb16f42d73caa47d3c9dc07eac15527d0f83028e9adf3d9e0eb421b14a"}
Jul 23 22:54:48.806323 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:48.806265    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" podStartSLOduration=1.8062439590000001 podStartE2EDuration="1.806243959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:54:48.805942438 +0000 UTC m=+1919.169608825" watchObservedRunningTime="2026-07-23 22:54:48.806243959 +0000 UTC m=+1919.169910343"
Jul 23 22:54:48.837768 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:48.837729    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 23 22:54:48.837937 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:48.837794    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 23 22:54:49.837682 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:49.837650    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 23 22:54:49.838071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:49.837706    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 23 22:54:50.592532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:50.592497    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:54:50.837716 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:50.837658    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 23 22:54:50.838141 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:50.837761    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 23 22:54:51.841910 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:51.841863    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:51.842306 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:51.842141    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:51.842929 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:51.842909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:54:57.828736 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:57.828694    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:57.829237 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:57.828889    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:57.842225 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:57.842198    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:54:58.850548 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:54:58.850520    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:55:08.191128 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:08.191096    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:55:29.800385 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:29.800345    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:55:40.796017 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:40.795937    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:55:44.413914 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:44.413871    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:55:59.434408 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.434340    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" podUID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" containerName="main" containerID="cri-o://938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af" gracePeriod=45
Jul 23 22:55:59.685638 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.685573    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:55:59.781303 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781303 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781316    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781400    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781423    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781451    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flk64\" (UniqueName: \"kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781559 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781472    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm\") pod \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\" (UID: \"4c54744d-4b39-4e05-b720-9ccf21cdbd97\") "
Jul 23 22:55:59.781771 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781672    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:55:59.781771 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781723    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:55:59.781878 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.781767    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home" (OuterVolumeSpecName: "home") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:55:59.783814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.783780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64" (OuterVolumeSpecName: "kube-api-access-flk64") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "kube-api-access-flk64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:55:59.783814 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.783790    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm" (OuterVolumeSpecName: "dshm") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:55:59.784071 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.783837    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c54744d-4b39-4e05-b720-9ccf21cdbd97" (UID: "4c54744d-4b39-4e05-b720-9ccf21cdbd97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:55:59.882520 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882482    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:55:59.882520 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882512    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:55:59.882520 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882521    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flk64\" (UniqueName: \"kubernetes.io/projected/4c54744d-4b39-4e05-b720-9ccf21cdbd97-kube-api-access-flk64\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:55:59.882768 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882531    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:55:59.882768 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882542    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:55:59.882768 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:55:59.882552    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c54744d-4b39-4e05-b720-9ccf21cdbd97-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:00.141538 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.141496    2559 generic.go:356] "Generic (PLEG): container finished" podID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" containerID="938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af" exitCode=0
Jul 23 22:56:00.141720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.141570    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"
Jul 23 22:56:00.141720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.141583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" event={"ID":"4c54744d-4b39-4e05-b720-9ccf21cdbd97","Type":"ContainerDied","Data":"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"}
Jul 23 22:56:00.141720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.141630    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll" event={"ID":"4c54744d-4b39-4e05-b720-9ccf21cdbd97","Type":"ContainerDied","Data":"999d5fdb16f42d73caa47d3c9dc07eac15527d0f83028e9adf3d9e0eb421b14a"}
Jul 23 22:56:00.141720 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.141647    2559 scope.go:117] "RemoveContainer" containerID="938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"
Jul 23 22:56:00.150163 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.150142    2559 scope.go:117] "RemoveContainer" containerID="938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"
Jul 23 22:56:00.150427 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:56:00.150407    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af\": container with ID starting with 938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af not found: ID does not exist" containerID="938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"
Jul 23 22:56:00.150474 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.150437    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af"} err="failed to get container status \"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af\": rpc error: code = NotFound desc = could not find container \"938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af\": container with ID starting with 938e107c8d220b197eef3f29a0b7318aaf5229f57d58f84adfe68a390d1016af not found: ID does not exist"
Jul 23 22:56:00.157296 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.157269    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:56:00.158805 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:00.158785    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b2ff72b5/e2e-lifecycle-model-b2ff72b5-kserve-6cb7bc668c-dghll"]
Jul 23 22:56:02.109692 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:02.109658    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" path="/var/lib/kubelet/pods/4c54744d-4b39-4e05-b720-9ccf21cdbd97/volumes"
Jul 23 22:56:33.702092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:33.702058    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:56:33.702700 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:33.702261    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" containerID="cri-o://37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221" gracePeriod=30
Jul 23 22:56:36.844977 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:36.844934    2559 patch_prober.go:28] interesting pod/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 22:56:36.845428 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:36.845003    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 22:56:37.402453 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:37.402409    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:56:38.952904 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:38.952866    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:56:39.032456 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032416    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032456 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032462    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032490    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t852d\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032513    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032535    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032651    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.032823 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032749    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:56:39.033167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032881    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:56:39.033167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033018    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:56:39.033167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.032684    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.033167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033063    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:56:39.033167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033131    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data\") pod \"2825e78a-1439-4017-b0a9-3452ab34f6b7\" (UID: \"2825e78a-1439-4017-b0a9-3452ab34f6b7\") "
Jul 23 22:56:39.033809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033749    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.033809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033778    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.033809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033792    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2825e78a-1439-4017-b0a9-3452ab34f6b7-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.033809 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.033808    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.034424 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.034396    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:56:39.034959 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.034930    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d" (OuterVolumeSpecName: "kube-api-access-t852d") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "kube-api-access-t852d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:56:39.035056 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.034980    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:56:39.035149 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.035134    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:56:39.035276 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.035250    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2825e78a-1439-4017-b0a9-3452ab34f6b7" (UID: "2825e78a-1439-4017-b0a9-3452ab34f6b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:56:39.134967 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.134933    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.134967 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.134961    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.134967 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.134970    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.135252 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.134979    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t852d\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-kube-api-access-t852d\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.135252 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.134988    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2825e78a-1439-4017-b0a9-3452ab34f6b7-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:56:39.328294 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.328203    2559 generic.go:356] "Generic (PLEG): container finished" podID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerID="37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221" exitCode=0
Jul 23 22:56:39.328294 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.328254    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"
Jul 23 22:56:39.328532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.328295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerDied","Data":"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"}
Jul 23 22:56:39.328532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.328361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr" event={"ID":"2825e78a-1439-4017-b0a9-3452ab34f6b7","Type":"ContainerDied","Data":"da1ae689a09abd6b8d2088a2fd55150684801696f173a75d2d82971f7d5f8331"}
Jul 23 22:56:39.328532 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.328392    2559 scope.go:117] "RemoveContainer" containerID="37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"
Jul 23 22:56:39.337851 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.337834    2559 scope.go:117] "RemoveContainer" containerID="d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"
Jul 23 22:56:39.345856 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.345838    2559 scope.go:117] "RemoveContainer" containerID="37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"
Jul 23 22:56:39.346131 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:56:39.346114    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221\": container with ID starting with 37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221 not found: ID does not exist" containerID="37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"
Jul 23 22:56:39.346207 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.346141    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221"} err="failed to get container status \"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221\": rpc error: code = NotFound desc = could not find container \"37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221\": container with ID starting with 37d511bcf87266780072e7a2853bf8a810f3bb2e1f8548d5ed6bad286e51d221 not found: ID does not exist"
Jul 23 22:56:39.346207 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.346157    2559 scope.go:117] "RemoveContainer" containerID="d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"
Jul 23 22:56:39.346400 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:56:39.346382    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9\": container with ID starting with d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9 not found: ID does not exist" containerID="d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"
Jul 23 22:56:39.346446 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.346407    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9"} err="failed to get container status \"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9\": rpc error: code = NotFound desc = could not find container \"d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9\": container with ID starting with d80b8511c3439d182df45075eefd5d89c4d278a1e041d7a821d128b68bd469a9 not found: ID does not exist"
Jul 23 22:56:39.360005 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.359984    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:56:39.380470 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:39.380432    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b2ff72b5-openshift-default-78d98f4b7-mbrrr"]
Jul 23 22:56:40.110197 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:40.110168    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" path="/var/lib/kubelet/pods/2825e78a-1439-4017-b0a9-3452ab34f6b7/volumes"
Jul 23 22:56:46.401453 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:46.401408    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:56:50.200916 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:56:50.200874    2559 scope.go:117] "RemoveContainer" containerID="5727a6bf0c437a11ef695a39570642c56642b50e2a46f446d160bd231ee2b2e3"
Jul 23 22:57:29.905109 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905026    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905370    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905384    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905401    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" containerName="main"
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905406    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" containerName="main"
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905481    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:29.905695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.905490    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c54744d-4b39-4e05-b720-9ccf21cdbd97" containerName="main"
Jul 23 22:57:29.908411 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.908394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.910481 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.910457    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c70c7c-openshift-default-dockercfg-rsdxr\""
Jul 23 22:57:29.921585 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.921559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 22:57:29.991625 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991582    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991625 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991629    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991725    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x8sn\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.991854 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991835    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.992094 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:29.992094 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:29.991985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092638 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092607    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092638 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x8sn\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092750    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.092906 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.092829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.093249 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.093106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.093249 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.093161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.093357 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.093276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.093357 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.093303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.093594 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.093577    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.095210 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.095191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.095331 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.095315    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.106453 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.106421    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x8sn\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.106590 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.106422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token\") pod \"e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") " pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.218131 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.218045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:30.358564 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.358528    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 22:57:30.362309 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:57:30.362281    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0c0b83_7b30_4fb9_8912_3210d47795db.slice/crio-c4c6a22d59b8431dc039bfaae9cf4d7594a08af28301616ff3d16641aac6c33c WatchSource:0}: Error finding container c4c6a22d59b8431dc039bfaae9cf4d7594a08af28301616ff3d16641aac6c33c: Status 404 returned error can't find the container with id c4c6a22d59b8431dc039bfaae9cf4d7594a08af28301616ff3d16641aac6c33c
Jul 23 22:57:30.364364 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.364328    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:30.364459 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.364395    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:30.364459 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.364424    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:30.575488 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.575444    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerStarted","Data":"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"}
Jul 23 22:57:30.575675 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.575519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerStarted","Data":"c4c6a22d59b8431dc039bfaae9cf4d7594a08af28301616ff3d16641aac6c33c"}
Jul 23 22:57:30.597150 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:30.597083    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podStartSLOduration=1.5970669819999999 podStartE2EDuration="1.597066982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:57:30.595741935 +0000 UTC m=+2080.959408344" watchObservedRunningTime="2026-07-23 22:57:30.597066982 +0000 UTC m=+2080.960733366"
Jul 23 22:57:31.218269 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:31.218231    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:31.223459 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:31.223433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:31.580688 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:31.580657    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:31.581596 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:31.581576    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:37.739068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.739024    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 22:57:37.739549 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.739528    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:37.739626 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.739553    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:37.739683 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.739660    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2825e78a-1439-4017-b0a9-3452ab34f6b7" containerName="istio-proxy"
Jul 23 22:57:37.742648 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.742627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.745204 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.745185    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-798605-openshift-default-dockercfg-zlvjt\""
Jul 23 22:57:37.754760 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.754740    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 22:57:37.868680 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.868855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.868855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.868855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z6zg\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.868855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868822    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.868855 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.869063 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.869063 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.868950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.869063 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.869031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970066 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z6zg\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970248    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970285    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970530 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970717 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970822 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970861 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.970992 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.970974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.973088 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.973069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.973341 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.973321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.978308 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.978278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:37.978987 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:37.978966    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z6zg\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg\") pod \"e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") " pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:38.053655 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.053559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:38.192223 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.192137    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 22:57:38.196082 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:57:38.196034    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d761a34_1abc_4218_8a48_3458c4ddcdaf.slice/crio-f6f24d0a49a7342b47c1dd51edda285d8c015e4633fa223c894b79cb34c47484 WatchSource:0}: Error finding container f6f24d0a49a7342b47c1dd51edda285d8c015e4633fa223c894b79cb34c47484: Status 404 returned error can't find the container with id f6f24d0a49a7342b47c1dd51edda285d8c015e4633fa223c894b79cb34c47484
Jul 23 22:57:38.198169 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.198129    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:38.198259 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.198208    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:38.198259 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.198246    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:38.605788 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.605753    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:57:38.615248 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.615221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerStarted","Data":"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"}
Jul 23 22:57:38.615405 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.615267    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerStarted","Data":"f6f24d0a49a7342b47c1dd51edda285d8c015e4633fa223c894b79cb34c47484"}
Jul 23 22:57:38.635250 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:38.635204    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podStartSLOduration=1.635187845 podStartE2EDuration="1.635187845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:57:38.634295729 +0000 UTC m=+2088.997962134" watchObservedRunningTime="2026-07-23 22:57:38.635187845 +0000 UTC m=+2088.998854283"
Jul 23 22:57:39.054076 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:39.054037    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:39.059028 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:39.059000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:39.621421 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:39.621369    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:39.622352 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:39.622326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:46.988695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:46.987627    2559 patch_prober.go:28] interesting pod/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:39870->10.133.0.40:15021: read: connection reset by peer" start-of-body=
Jul 23 22:57:46.988695 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:46.987703    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:39870->10.133.0.40:15021: read: connection reset by peer"
Jul 23 22:57:47.660875 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.660843    2559 generic.go:356] "Generic (PLEG): container finished" podID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerID="83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e" exitCode=0
Jul 23 22:57:47.661069 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.660916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerDied","Data":"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"}
Jul 23 22:57:47.661312 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.661300    2559 scope.go:117] "RemoveContainer" containerID="83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"
Jul 23 22:57:47.662335 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.662303    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:47.662388 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.662378    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:47.662426 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:47.662412    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:48.668850 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:48.668807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerStarted","Data":"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"}
Jul 23 22:57:49.218846 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:49.218790    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:49.219644 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:49.219322    2559 patch_prober.go:28] interesting pod/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 23 22:57:49.219644 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:49.219380    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 23 22:57:50.219496 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.219448    2559 patch_prober.go:28] interesting pod/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 23 22:57:50.219905 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.219541    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 23 22:57:50.226507 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.226458    2559 scope.go:117] "RemoveContainer" containerID="d41f8d5d7a8a52457b3a090f516a1f289260a4150e258cc241a5fea777a33391"
Jul 23 22:57:50.237396 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.237370    2559 scope.go:117] "RemoveContainer" containerID="6ec4a292b17538276e2cb62d21f3b6478a4c5d331e405ae95b2698bca777031b"
Jul 23 22:57:50.802002 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.801944    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:57:50.818590 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.818546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:57:50.818781 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.818674    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.822723 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.822685    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c70c7c\"/\"kube-root-ca.crt\""
Jul 23 22:57:50.823174 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.822686    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c70c7c\"/\"default-dockercfg-4tgtw\""
Jul 23 22:57:50.823174 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.823124    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c70c7c\"/\"openshift-service-ca.crt\""
Jul 23 22:57:50.823174 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.823142    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c70c7c\"/\"test-model-c70c7c-kserve-self-signed-certs\""
Jul 23 22:57:50.918500 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.918692 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918532    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.918692 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.918808 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.918808 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:50.918952 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:50.918853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8kg\" (UniqueName: \"kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.019734    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.019807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.019856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8kg\" (UniqueName: \"kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.019972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.020021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.020059    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.020523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.021220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.022928 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.021475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.023916 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.023869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.024951 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.024926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.031345 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.031311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8kg\" (UniqueName: \"kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg\") pod \"test-model-c70c7c-kserve-6b4df6c7ff-42d42\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") " pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.130584 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.130491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:57:51.222933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.220326    2559 patch_prober.go:28] interesting pod/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 23 22:57:51.222933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.220392    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 23 22:57:51.289494 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.289468    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:57:51.291947 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:57:51.291911    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ac446d5_d029_4e88_9407_91224d6bde84.slice/crio-aca70686458ff3f96e3cab32bce54fbf3ed307be44d605d4fd357ad9406ec38b WatchSource:0}: Error finding container aca70686458ff3f96e3cab32bce54fbf3ed307be44d605d4fd357ad9406ec38b: Status 404 returned error can't find the container with id aca70686458ff3f96e3cab32bce54fbf3ed307be44d605d4fd357ad9406ec38b
Jul 23 22:57:51.322031 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.321982    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:57:51.685741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.685655    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" event={"ID":"3ac446d5-d029-4e88-9407-91224d6bde84","Type":"ContainerStarted","Data":"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"}
Jul 23 22:57:51.685741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:51.685700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" event={"ID":"3ac446d5-d029-4e88-9407-91224d6bde84","Type":"ContainerStarted","Data":"aca70686458ff3f96e3cab32bce54fbf3ed307be44d605d4fd357ad9406ec38b"}
Jul 23 22:57:52.219484 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:52.219444    2559 patch_prober.go:28] interesting pod/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 23 22:57:52.219663 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:52.219532    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 23 22:57:53.222655 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.222624    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:53.223172 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.222999    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:53.223748 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.223729    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:57:53.244049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.244000    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" podStartSLOduration=3.243984632 podStartE2EDuration="3.243984632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:57:51.70524999 +0000 UTC m=+2102.068916374" watchObservedRunningTime="2026-07-23 22:57:53.243984632 +0000 UTC m=+2103.607651015"
Jul 23 22:57:53.696960 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.696846    2559 generic.go:356] "Generic (PLEG): container finished" podID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerID="89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f" exitCode=0
Jul 23 22:57:53.696960 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.696922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerDied","Data":"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"}
Jul 23 22:57:53.697464 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.697444    2559 scope.go:117] "RemoveContainer" containerID="89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"
Jul 23 22:57:53.698435 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.698410    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:53.698485 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.698476    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:53.698535 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:53.698523    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 22:57:54.621421 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:54.621379    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:54.708122 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:54.708046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerStarted","Data":"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"}
Jul 23 22:57:55.054527 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:55.054482    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:55.055751 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:55.055725    2559 patch_prober.go:28] interesting pod/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 23 22:57:55.055858 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:55.055775    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 23 22:57:56.054834 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:56.054791    2559 patch_prober.go:28] interesting pod/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 23 22:57:56.055269 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:56.054855    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 23 22:57:57.053873 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:57.053838    2559 patch_prober.go:28] interesting pod/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 23 22:57:57.054064 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:57.053912    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 23 22:57:58.054997 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:58.054962    2559 patch_prober.go:28] interesting pod/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 23 22:57:58.055426 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:58.055023    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 23 22:57:59.058699 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:59.058668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:59.059124 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:59.058997    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:57:59.059741 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:57:59.059725    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 22:58:00.998697 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:00.998660    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:58:01.131445 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:01.131414    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:58:01.131630 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:01.131463    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:58:01.143945 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:01.143914    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:58:01.757057 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:01.757031    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:58:36.981639 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.981595    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:58:36.985690 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.985667    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:36.989458 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.989428    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-798605\"/\"openshift-service-ca.crt\""
Jul 23 22:58:36.989458 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.989449    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-798605\"/\"default-dockercfg-xtdvf\""
Jul 23 22:58:36.989678 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.989527    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-798605\"/\"test-model-798605-kserve-self-signed-certs\""
Jul 23 22:58:36.989678 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.989568    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-798605\"/\"kube-root-ca.crt\""
Jul 23 22:58:36.995264 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:36.995244    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:58:37.061319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.061319 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061324    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.061547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66xx9\" (UniqueName: \"kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.061547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.061547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.061547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.061499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.162715 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.162677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.162933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.162735    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.162933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.162776    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.162933 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.162834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.163068 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.163028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.163113 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.163074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66xx9\" (UniqueName: \"kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.163176 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.163147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.163229 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.163178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.163343 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.163325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.165281 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.165261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.165474 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.165457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.171942 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.171919    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66xx9\" (UniqueName: \"kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9\") pod \"test-model-798605-kserve-76df776f55-5fdq8\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") " pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.297487 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.297387    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:37.430688 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.430664    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:58:37.434696 ip-10-0-133-188 kubenswrapper[2559]: W0723 22:58:37.434664    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a35d871_4aac_40cc_be08_6e14fccfc6b7.slice/crio-9db5d7b7cfd0a8f53310079aba74e26d88328a81c391600e29fc14daacacb471 WatchSource:0}: Error finding container 9db5d7b7cfd0a8f53310079aba74e26d88328a81c391600e29fc14daacacb471: Status 404 returned error can't find the container with id 9db5d7b7cfd0a8f53310079aba74e26d88328a81c391600e29fc14daacacb471
Jul 23 22:58:37.913491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.913446    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" event={"ID":"7a35d871-4aac-40cc-be08-6e14fccfc6b7","Type":"ContainerStarted","Data":"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"}
Jul 23 22:58:37.913671 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.913501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" event={"ID":"7a35d871-4aac-40cc-be08-6e14fccfc6b7","Type":"ContainerStarted","Data":"9db5d7b7cfd0a8f53310079aba74e26d88328a81c391600e29fc14daacacb471"}
Jul 23 22:58:37.930547 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:37.930488    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" podStartSLOduration=1.93046915 podStartE2EDuration="1.93046915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:58:37.929506327 +0000 UTC m=+2148.293172836" watchObservedRunningTime="2026-07-23 22:58:37.93046915 +0000 UTC m=+2148.294135534"
Jul 23 22:58:42.296149 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:42.296114    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:58:47.297888 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:47.297847    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:47.298354 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:47.297948    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:47.311033 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:47.311002    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:58:47.978148 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:58:47.978114    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:59:30.517316 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:30.517283    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:59:31.436049 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:31.436007    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:59:37.892596 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:37.892560    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:59:45.537837 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.537791    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" podUID="3ac446d5-d029-4e88-9407-91224d6bde84" containerName="main" containerID="cri-o://475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678" gracePeriod=45
Jul 23 22:59:45.783148 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.783122    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:59:45.837871 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.837791    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.837871 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.837847    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.837871 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.837868    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.838167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.837928    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q8kg\" (UniqueName: \"kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.838167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.837970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.838167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm\") pod \"3ac446d5-d029-4e88-9407-91224d6bde84\" (UID: \"3ac446d5-d029-4e88-9407-91224d6bde84\") "
Jul 23 22:59:45.838167 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838131    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home" (OuterVolumeSpecName: "home") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:45.838367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838147    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:45.838367 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838248    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:45.838467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838403    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:45.838467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838422    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:45.838467 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.838438    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:45.840283 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.840255    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:59:45.840376 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.840281    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm" (OuterVolumeSpecName: "dshm") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:45.840376 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.840351    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg" (OuterVolumeSpecName: "kube-api-access-5q8kg") pod "3ac446d5-d029-4e88-9407-91224d6bde84" (UID: "3ac446d5-d029-4e88-9407-91224d6bde84"). InnerVolumeSpecName "kube-api-access-5q8kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:59:45.939225 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.939195    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ac446d5-d029-4e88-9407-91224d6bde84-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:45.939225 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.939223    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q8kg\" (UniqueName: \"kubernetes.io/projected/3ac446d5-d029-4e88-9407-91224d6bde84-kube-api-access-5q8kg\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:45.939424 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:45.939235    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ac446d5-d029-4e88-9407-91224d6bde84-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.240726 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.240691    2559 generic.go:356] "Generic (PLEG): container finished" podID="3ac446d5-d029-4e88-9407-91224d6bde84" containerID="475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678" exitCode=0
Jul 23 22:59:46.240924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.240738    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"
Jul 23 22:59:46.240924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.240779    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" event={"ID":"3ac446d5-d029-4e88-9407-91224d6bde84","Type":"ContainerDied","Data":"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"}
Jul 23 22:59:46.240924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.240822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42" event={"ID":"3ac446d5-d029-4e88-9407-91224d6bde84","Type":"ContainerDied","Data":"aca70686458ff3f96e3cab32bce54fbf3ed307be44d605d4fd357ad9406ec38b"}
Jul 23 22:59:46.240924 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.240843    2559 scope.go:117] "RemoveContainer" containerID="475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"
Jul 23 22:59:46.249612 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.249592    2559 scope.go:117] "RemoveContainer" containerID="475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"
Jul 23 22:59:46.249878 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:59:46.249861    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678\": container with ID starting with 475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678 not found: ID does not exist" containerID="475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"
Jul 23 22:59:46.249936 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.249888    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678"} err="failed to get container status \"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678\": rpc error: code = NotFound desc = could not find container \"475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678\": container with ID starting with 475a4b5963da3c2cd10e55cd440e95b284818a176b717677470de83e7ef2f678 not found: ID does not exist"
Jul 23 22:59:46.258531 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.258508    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:59:46.266725 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.266705    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c70c7c/test-model-c70c7c-kserve-6b4df6c7ff-42d42"]
Jul 23 22:59:46.456702 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.456607    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" podUID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" containerName="main" containerID="cri-o://1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160" gracePeriod=45
Jul 23 22:59:46.594705 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.594658    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 22:59:46.712108 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.712044    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:59:46.847114 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847083    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847120    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847148    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66xx9\" (UniqueName: \"kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847258 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847215    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847471 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847295    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir\") pod \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\" (UID: \"7a35d871-4aac-40cc-be08-6e14fccfc6b7\") "
Jul 23 22:59:46.847471 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home" (OuterVolumeSpecName: "home") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:46.847471 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847460    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:46.847631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847607    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.847631 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847605    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:46.847730 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.847619    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.849660 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.849639    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:59:46.849751 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.849682    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm" (OuterVolumeSpecName: "dshm") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:46.849751 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.849712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9" (OuterVolumeSpecName: "kube-api-access-66xx9") pod "7a35d871-4aac-40cc-be08-6e14fccfc6b7" (UID: "7a35d871-4aac-40cc-be08-6e14fccfc6b7"). InnerVolumeSpecName "kube-api-access-66xx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:59:46.948954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.948915    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.948954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.948948    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66xx9\" (UniqueName: \"kubernetes.io/projected/7a35d871-4aac-40cc-be08-6e14fccfc6b7-kube-api-access-66xx9\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.948954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.948956    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a35d871-4aac-40cc-be08-6e14fccfc6b7-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:46.948954 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:46.948965    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a35d871-4aac-40cc-be08-6e14fccfc6b7-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:47.248922 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.248857    2559 generic.go:356] "Generic (PLEG): container finished" podID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" containerID="1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160" exitCode=0
Jul 23 22:59:47.249099 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.248999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" event={"ID":"7a35d871-4aac-40cc-be08-6e14fccfc6b7","Type":"ContainerDied","Data":"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"}
Jul 23 22:59:47.249099 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.249047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8" event={"ID":"7a35d871-4aac-40cc-be08-6e14fccfc6b7","Type":"ContainerDied","Data":"9db5d7b7cfd0a8f53310079aba74e26d88328a81c391600e29fc14daacacb471"}
Jul 23 22:59:47.249099 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.249071    2559 scope.go:117] "RemoveContainer" containerID="1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"
Jul 23 22:59:47.249265 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.249252    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"
Jul 23 22:59:47.259923 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.259862    2559 scope.go:117] "RemoveContainer" containerID="1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"
Jul 23 22:59:47.260295 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:59:47.260269    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160\": container with ID starting with 1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160 not found: ID does not exist" containerID="1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"
Jul 23 22:59:47.260378 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.260300    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160"} err="failed to get container status \"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160\": rpc error: code = NotFound desc = could not find container \"1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160\": container with ID starting with 1742ca3a3500fcaf17f2419230177b1a98e868ae657df547b93520ccdbed0160 not found: ID does not exist"
Jul 23 22:59:47.272598 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.272574    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:59:47.275543 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:47.275519    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-798605/test-model-798605-kserve-76df776f55-5fdq8"]
Jul 23 22:59:48.110006 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:48.109971    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ac446d5-d029-4e88-9407-91224d6bde84" path="/var/lib/kubelet/pods/3ac446d5-d029-4e88-9407-91224d6bde84/volumes"
Jul 23 22:59:48.110376 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:48.110366    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" path="/var/lib/kubelet/pods/7a35d871-4aac-40cc-be08-6e14fccfc6b7/volumes"
Jul 23 22:59:50.269512 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:50.269477    2559 scope.go:117] "RemoveContainer" containerID="133040e45ae9f381bb757709aae26261ec40cb48742ab7b1e66f41631fd2842b"
Jul 23 22:59:53.267489 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:53.267453    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 22:59:53.267860 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:53.267775    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy" containerID="cri-o://4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae" gracePeriod=30
Jul 23 22:59:58.522117 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.522090    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:59:58.662383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662297    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662340    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662366    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662383 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662382    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662402    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662441    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x8sn\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662463    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662497    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662527    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs\") pod \"9a0c0b83-7b30-4fb9-8912-3210d47795db\" (UID: \"9a0c0b83-7b30-4fb9-8912-3210d47795db\") "
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662669    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:58.662703 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:58.663092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662862    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.663092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662880    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.663092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.662927    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:58.663092 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.663013    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:59:58.663608 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.663582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:58.664911 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.664872    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn" (OuterVolumeSpecName: "kube-api-access-7x8sn") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "kube-api-access-7x8sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:59:58.665004 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.664965    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:59:58.665054 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.665009    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:59:58.665093 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.665081    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a0c0b83-7b30-4fb9-8912-3210d47795db" (UID: "9a0c0b83-7b30-4fb9-8912-3210d47795db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 22:59:58.765239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765199    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765239 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765240    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0c0b83-7b30-4fb9-8912-3210d47795db-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765262    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765273    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765283    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765298    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0c0b83-7b30-4fb9-8912-3210d47795db-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:58.765491 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:58.765317    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x8sn\" (UniqueName: \"kubernetes.io/projected/9a0c0b83-7b30-4fb9-8912-3210d47795db-kube-api-access-7x8sn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 22:59:59.312669 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.312635    2559 generic.go:356] "Generic (PLEG): container finished" podID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerID="4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae" exitCode=0
Jul 23 22:59:59.312841 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.312690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerDied","Data":"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"}
Jul 23 22:59:59.312841 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.312704    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"
Jul 23 22:59:59.312841 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.312741    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv" event={"ID":"9a0c0b83-7b30-4fb9-8912-3210d47795db","Type":"ContainerDied","Data":"c4c6a22d59b8431dc039bfaae9cf4d7594a08af28301616ff3d16641aac6c33c"}
Jul 23 22:59:59.312841 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.312765    2559 scope.go:117] "RemoveContainer" containerID="4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"
Jul 23 22:59:59.322907 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.322879    2559 scope.go:117] "RemoveContainer" containerID="83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"
Jul 23 22:59:59.331010 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.330991    2559 scope.go:117] "RemoveContainer" containerID="4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"
Jul 23 22:59:59.331296 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:59:59.331277    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae\": container with ID starting with 4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae not found: ID does not exist" containerID="4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"
Jul 23 22:59:59.331349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.331307    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae"} err="failed to get container status \"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae\": rpc error: code = NotFound desc = could not find container \"4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae\": container with ID starting with 4d17ec91499f23d0c0f9c2d59ff9f557515c0433f3dec91a7535948da46b47ae not found: ID does not exist"
Jul 23 22:59:59.331349 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.331325    2559 scope.go:117] "RemoveContainer" containerID="83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"
Jul 23 22:59:59.331536 ip-10-0-133-188 kubenswrapper[2559]: E0723 22:59:59.331524    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e\": container with ID starting with 83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e not found: ID does not exist" containerID="83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"
Jul 23 22:59:59.331579 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.331542    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e"} err="failed to get container status \"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e\": rpc error: code = NotFound desc = could not find container \"83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e\": container with ID starting with 83c1512bdeea678a98be34125ade075ee899a3a246d1350309106f6d7ae71c4e not found: ID does not exist"
Jul 23 22:59:59.334614 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.334587    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 22:59:59.336577 ip-10-0-133-188 kubenswrapper[2559]: I0723 22:59:59.336557    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c70c7c-openshift-default-75dcc8df95-r8ckv"]
Jul 23 23:00:00.110563 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.110528    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" path="/var/lib/kubelet/pods/9a0c0b83-7b30-4fb9-8912-3210d47795db/volumes"
Jul 23 23:00:00.136993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.136953    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"]
Jul 23 23:00:00.137520 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137493    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ac446d5-d029-4e88-9407-91224d6bde84" containerName="main"
Jul 23 23:00:00.137520 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137521    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ac446d5-d029-4e88-9407-91224d6bde84" containerName="main"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137545    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137555    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137574    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" containerName="main"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137582    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" containerName="main"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137674    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137686    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a35d871-4aac-40cc-be08-6e14fccfc6b7" containerName="main"
Jul 23 23:00:00.137730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.137696    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ac446d5-d029-4e88-9407-91224d6bde84" containerName="main"
Jul 23 23:00:00.142527 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.142499    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"]
Jul 23 23:00:00.142668 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.142646    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"
Jul 23 23:00:00.142994 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.142973    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.142994 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.142995    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.143158 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.143078    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a0c0b83-7b30-4fb9-8912-3210d47795db" containerName="istio-proxy"
Jul 23 23:00:00.145614 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.145588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:00:00.146071 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.146052    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"
Jul 23 23:00:00.147283 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.147263    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"]
Jul 23 23:00:00.150244 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.150218    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"]
Jul 23 23:00:00.158319 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.158292    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"
Jul 23 23:00:00.165911 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.165873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"
Jul 23 23:00:00.302718 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.302688    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"]
Jul 23 23:00:00.305022 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:00:00.304992    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341890ab_3139_4bec_9421_2bea54f94d1d.slice/crio-7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb WatchSource:0}: Error finding container 7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb: Status 404 returned error can't find the container with id 7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb
Jul 23 23:00:00.319031 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.319002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f" event={"ID":"341890ab-3139-4bec-9421-2bea54f94d1d","Type":"ContainerStarted","Data":"7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb"}
Jul 23 23:00:00.329974 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:00.329940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"]
Jul 23 23:00:00.343215 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:00:00.343181    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ea1a06_d144_4229_b389_92c33d7ff0e9.slice/crio-1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8 WatchSource:0}: Error finding container 1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8: Status 404 returned error can't find the container with id 1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8
Jul 23 23:00:01.325504 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:01.325461    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n" event={"ID":"b0ea1a06-d144-4229-b389-92c33d7ff0e9","Type":"ContainerStarted","Data":"b2273586c6dc7059e2a9b1e7e45e9434144fb26dd1a4f76caf06a4cdaf3da538"}
Jul 23 23:00:01.326088 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:01.325530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n" event={"ID":"b0ea1a06-d144-4229-b389-92c33d7ff0e9","Type":"ContainerStarted","Data":"1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8"}
Jul 23 23:00:01.327025 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:01.327001    2559 generic.go:356] "Generic (PLEG): container finished" podID="341890ab-3139-4bec-9421-2bea54f94d1d" containerID="8f5fb81d6656c6a4dd025037c7528a3638e3768acf6006b132c3ce7d94e2dcb2" exitCode=0
Jul 23 23:00:01.327124 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:01.327081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f" event={"ID":"341890ab-3139-4bec-9421-2bea54f94d1d","Type":"ContainerDied","Data":"8f5fb81d6656c6a4dd025037c7528a3638e3768acf6006b132c3ce7d94e2dcb2"}
Jul 23 23:00:01.341363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:01.341291    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n" podStartSLOduration=1.341268175 podStartE2EDuration="1.341268175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:00:01.341061789 +0000 UTC m=+2231.704728188" watchObservedRunningTime="2026-07-23 23:00:01.341268175 +0000 UTC m=+2231.704934558"
Jul 23 23:00:02.333090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:02.333053    2559 generic.go:356] "Generic (PLEG): container finished" podID="b0ea1a06-d144-4229-b389-92c33d7ff0e9" containerID="b2273586c6dc7059e2a9b1e7e45e9434144fb26dd1a4f76caf06a4cdaf3da538" exitCode=0
Jul 23 23:00:02.333532 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:02.333136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n" event={"ID":"b0ea1a06-d144-4229-b389-92c33d7ff0e9","Type":"ContainerDied","Data":"b2273586c6dc7059e2a9b1e7e45e9434144fb26dd1a4f76caf06a4cdaf3da538"}
Jul 23 23:00:02.472912 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:02.472875    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"
Jul 23 23:00:03.339026 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:03.338987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f" event={"ID":"341890ab-3139-4bec-9421-2bea54f94d1d","Type":"ContainerDied","Data":"7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb"}
Jul 23 23:00:03.339458 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:03.339054    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c015c77c2fe8e8792aa3ee2295d656e35ac491d29f19a9996e0cf5fa8e48dcb"
Jul 23 23:00:03.339458 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:03.339004    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"
Jul 23 23:00:03.476078 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:03.476051    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"
Jul 23 23:00:04.011493 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.011452    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"]
Jul 23 23:00:04.013474 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.013449    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-798605-29747460-7mw4f"]
Jul 23 23:00:04.110005 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.109974    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341890ab-3139-4bec-9421-2bea54f94d1d" path="/var/lib/kubelet/pods/341890ab-3139-4bec-9421-2bea54f94d1d/volumes"
Jul 23 23:00:04.345081 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.344993    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n" event={"ID":"b0ea1a06-d144-4229-b389-92c33d7ff0e9","Type":"ContainerDied","Data":"1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8"}
Jul 23 23:00:04.345081 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.345049    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e1b530f81f93a08bf609764593cd8bc8d34353abf1caeb60ba2745f450b5ce8"
Jul 23 23:00:04.345081 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:04.345011    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747460-nb29n"
Jul 23 23:00:06.794156 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:06.794118    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:00:15.368625 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:15.368541    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 23:00:15.369068 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:15.368813    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy" containerID="cri-o://a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4" gracePeriod=30
Jul 23 23:00:16.261202 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261168    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:00:16.261553 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261540    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ea1a06-d144-4229-b389-92c33d7ff0e9" containerName="cleanup"
Jul 23 23:00:16.261603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261555    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ea1a06-d144-4229-b389-92c33d7ff0e9" containerName="cleanup"
Jul 23 23:00:16.261603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261567    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341890ab-3139-4bec-9421-2bea54f94d1d" containerName="cleanup"
Jul 23 23:00:16.261603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261573    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="341890ab-3139-4bec-9421-2bea54f94d1d" containerName="cleanup"
Jul 23 23:00:16.261698 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261632    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="341890ab-3139-4bec-9421-2bea54f94d1d" containerName="cleanup"
Jul 23 23:00:16.261698 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.261644    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0ea1a06-d144-4229-b389-92c33d7ff0e9" containerName="cleanup"
Jul 23 23:00:16.264910 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.264876    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.267451 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.267429    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-70930e-openshift-default-dockercfg-8mqwd\""
Jul 23 23:00:16.276113 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.276088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:00:16.330538 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330626    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330714    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330980 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330980 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6mm4\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.330980 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.330810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431533 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431551    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431604    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6mm4\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.431983 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.431777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.432344 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.432047    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.432344 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.432069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.432344 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.432257    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.432344 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.432307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.432971 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.432948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.434183 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.434162    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.434380 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.434365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.439972 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.439941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.440260 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.440237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6mm4\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4\") pod \"e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") " pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.575478 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.575388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:16.740383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.740358    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:00:16.743019 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:00:16.742970    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e9200b_c144_48bb_b1f1_654aa5bf4edf.slice/crio-f9c092035504eca17a158b0b8372892e476209e1b9c764e4981005a4869141b6 WatchSource:0}: Error finding container f9c092035504eca17a158b0b8372892e476209e1b9c764e4981005a4869141b6: Status 404 returned error can't find the container with id f9c092035504eca17a158b0b8372892e476209e1b9c764e4981005a4869141b6
Jul 23 23:00:16.745433 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.745388    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:16.745528 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.745477    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:16.745528 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:16.745519    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:17.407147 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:17.407111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerStarted","Data":"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"}
Jul 23 23:00:17.407336 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:17.407166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerStarted","Data":"f9c092035504eca17a158b0b8372892e476209e1b9c764e4981005a4869141b6"}
Jul 23 23:00:17.426469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:17.426398    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podStartSLOduration=1.426378671 podStartE2EDuration="1.426378671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:00:17.426330465 +0000 UTC m=+2247.789996860" watchObservedRunningTime="2026-07-23 23:00:17.426378671 +0000 UTC m=+2247.790045055"
Jul 23 23:00:17.575674 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:17.575637    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:17.580348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:17.580324    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:18.411704 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:18.411679    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:18.412553 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:18.412532    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:20.630577 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.630514    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 23:00:20.673389 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673356    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673420    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673487    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673549    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z6zg\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673565    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673596    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673626    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673650    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token\") pod \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\" (UID: \"2d761a34-1abc-4218-8a48-3458c4ddcdaf\") "
Jul 23 23:00:20.673855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673690    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:00:20.673855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:00:20.674160 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.673998    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:00:20.674160 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.674020    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.674160 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.674040    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.674349 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.674321    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:00:20.674721 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.674616    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:00:20.676171 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.676140    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:00:20.676292 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.676210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg" (OuterVolumeSpecName: "kube-api-access-8z6zg") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "kube-api-access-8z6zg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:00:20.676423 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.676405    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:00:20.676487 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.676472    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d761a34-1abc-4218-8a48-3458c4ddcdaf" (UID: "2d761a34-1abc-4218-8a48-3458c4ddcdaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:00:20.774759 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774716    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.774759 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774752    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.774759 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774765    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z6zg\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-kube-api-access-8z6zg\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.775048 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774779    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.775048 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774790    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.775048 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774802    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:20.775048 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:20.774812    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d761a34-1abc-4218-8a48-3458c4ddcdaf-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:21.428201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.428165    2559 generic.go:356] "Generic (PLEG): container finished" podID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerID="a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4" exitCode=0
Jul 23 23:00:21.428401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.428216    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerDied","Data":"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"}
Jul 23 23:00:21.428401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.428225    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"
Jul 23 23:00:21.428401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.428259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5" event={"ID":"2d761a34-1abc-4218-8a48-3458c4ddcdaf","Type":"ContainerDied","Data":"f6f24d0a49a7342b47c1dd51edda285d8c015e4633fa223c894b79cb34c47484"}
Jul 23 23:00:21.428401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.428285    2559 scope.go:117] "RemoveContainer" containerID="a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"
Jul 23 23:00:21.437251 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.437235    2559 scope.go:117] "RemoveContainer" containerID="89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"
Jul 23 23:00:21.445072 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.445050    2559 scope.go:117] "RemoveContainer" containerID="a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"
Jul 23 23:00:21.445326 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:00:21.445308    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4\": container with ID starting with a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4 not found: ID does not exist" containerID="a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"
Jul 23 23:00:21.445392 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.445334    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4"} err="failed to get container status \"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4\": rpc error: code = NotFound desc = could not find container \"a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4\": container with ID starting with a9fe091919f5991d1bb2de14310992280ba00675dde98c18273b00c36fa2bda4 not found: ID does not exist"
Jul 23 23:00:21.445392 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.445351    2559 scope.go:117] "RemoveContainer" containerID="89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"
Jul 23 23:00:21.445564 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:00:21.445547    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f\": container with ID starting with 89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f not found: ID does not exist" containerID="89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"
Jul 23 23:00:21.445600 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.445571    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f"} err="failed to get container status \"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f\": rpc error: code = NotFound desc = could not find container \"89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f\": container with ID starting with 89fe81f921999c6bc3d790345564e1611a8df4e40d0d220a8601c568e34a595f not found: ID does not exist"
Jul 23 23:00:21.448691 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.448670    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 23:00:21.451886 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:21.451864    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-798605-openshift-default-56bdc9cc6f-r5wz5"]
Jul 23 23:00:22.110389 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:22.110360    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" path="/var/lib/kubelet/pods/2d761a34-1abc-4218-8a48-3458c4ddcdaf/volumes"
Jul 23 23:00:24.638787 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.638746    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:00:24.639307 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639288    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.639387 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639310    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.639387 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639330    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.639387 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639340    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.639532 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639425    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.639532 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.639439    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d761a34-1abc-4218-8a48-3458c4ddcdaf" containerName="istio-proxy"
Jul 23 23:00:24.642265 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.642243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.644559 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.644540    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-857bc8-openshift-default-dockercfg-2fcpp\""
Jul 23 23:00:24.653865 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.653842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:00:24.713186 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n9qf\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713186 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713352    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713437 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.713698 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.713484    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814097 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9qf\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814682 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814747 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814747 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814842 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814842 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.814944 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.814878    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.816932 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.816881    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.817062 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.817041    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.821560 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.821537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.821773 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.821756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9qf\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf\") pod \"e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") " pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:24.952003 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:24.951924    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:25.087738 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.087713    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:00:25.090086 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:00:25.090057    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b68eef_2a07_4da6_9fd9_2c4a29f12743.slice/crio-84ba02d46dbf83351045ea00f8eddf12594daae5f72790e7fc948cbd71e7cdbd WatchSource:0}: Error finding container 84ba02d46dbf83351045ea00f8eddf12594daae5f72790e7fc948cbd71e7cdbd: Status 404 returned error can't find the container with id 84ba02d46dbf83351045ea00f8eddf12594daae5f72790e7fc948cbd71e7cdbd
Jul 23 23:00:25.092116 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.092083    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:25.092200 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.092151    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:25.092200 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.092178    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:25.448311 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.448273    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerStarted","Data":"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"}
Jul 23 23:00:25.448512 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.448337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerStarted","Data":"84ba02d46dbf83351045ea00f8eddf12594daae5f72790e7fc948cbd71e7cdbd"}
Jul 23 23:00:25.468486 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.468434    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podStartSLOduration=1.468416865 podStartE2EDuration="1.468416865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:00:25.466394475 +0000 UTC m=+2255.830060858" watchObservedRunningTime="2026-07-23 23:00:25.468416865 +0000 UTC m=+2255.832083247"
Jul 23 23:00:25.952985 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:25.952947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:26.687987 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:26.687950    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:00:26.957577 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:26.957500    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:27.457697 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:27.457664    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:27.458791 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:27.458769    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:34.453115 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:34.453056    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 23:00:34.453573 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:34.453142    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 23:00:35.497856 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.497814    2559 generic.go:356] "Generic (PLEG): container finished" podID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerID="45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835" exitCode=0
Jul 23 23:00:35.498407 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.497874    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerDied","Data":"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"}
Jul 23 23:00:35.498503 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.498482    2559 scope.go:117] "RemoveContainer" containerID="45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"
Jul 23 23:00:35.499909 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.499853    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:35.500025 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.499962    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:35.500025 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:35.500011    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:36.510007 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:36.509961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerStarted","Data":"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"}
Jul 23 23:00:36.576888 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:36.576859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:36.577249 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:36.577172    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:00:36.577415 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:36.577395    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 23 23:00:37.576221 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:37.576181    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:00:37.576610 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:37.576235    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 23 23:00:38.205339 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:38.205294    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:00:38.576213 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:38.576175    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:00:38.576379 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:38.576232    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 23 23:00:39.576314 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:39.576275    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:00:39.576495 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:39.576338    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 23 23:00:40.121769 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.121735    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"]
Jul 23 23:00:40.124082 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.124060    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:40.126265 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.126240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:00:40.126391 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.126289    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:00:40.132606 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.132560    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"]
Jul 23 23:00:40.276504 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.276471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-70930e-4gkn9\" (UID: \"bc6d716f-69e1-4254-8664-f8ea2a3871ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:40.377499 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.377404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-70930e-4gkn9\" (UID: \"bc6d716f-69e1-4254-8664-f8ea2a3871ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:40.378168 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.378142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-70930e-4gkn9\" (UID: \"bc6d716f-69e1-4254-8664-f8ea2a3871ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:40.436544 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.436501    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:40.580182 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.580151    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:40.580561 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.580538    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:40.581190 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.581172    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:00:40.584762 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:40.584740    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"]
Jul 23 23:00:40.588099 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:00:40.588076    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6d716f_69e1_4254_8664_f8ea2a3871ab.slice/crio-c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1 WatchSource:0}: Error finding container c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1: Status 404 returned error can't find the container with id c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1
Jul 23 23:00:41.539616 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.539523    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc6d716f-69e1-4254-8664-f8ea2a3871ab" containerID="f072362f9d400b2d8728c8a7e10a2facd7b6988823903923090753a04c8eb561" exitCode=0
Jul 23 23:00:41.539616 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.539601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9" event={"ID":"bc6d716f-69e1-4254-8664-f8ea2a3871ab","Type":"ContainerDied","Data":"f072362f9d400b2d8728c8a7e10a2facd7b6988823903923090753a04c8eb561"}
Jul 23 23:00:41.539886 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.539657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9" event={"ID":"bc6d716f-69e1-4254-8664-f8ea2a3871ab","Type":"ContainerStarted","Data":"c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1"}
Jul 23 23:00:41.541178 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.541157    2559 generic.go:356] "Generic (PLEG): container finished" podID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerID="bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b" exitCode=0
Jul 23 23:00:41.541286 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.541234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerDied","Data":"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"}
Jul 23 23:00:41.541620 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.541605    2559 scope.go:117] "RemoveContainer" containerID="bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"
Jul 23 23:00:41.542661 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.542636    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:41.542715 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.542699    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:41.542755 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:41.542733    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:00:42.458018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.457978    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:42.548428 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.548391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerStarted","Data":"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"}
Jul 23 23:00:42.723469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.723443    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:42.915519 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.915476    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca\") pod \"bc6d716f-69e1-4254-8664-f8ea2a3871ab\" (UID: \"bc6d716f-69e1-4254-8664-f8ea2a3871ab\") "
Jul 23 23:00:42.916110 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.916077    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc6d716f-69e1-4254-8664-f8ea2a3871ab" (UID: "bc6d716f-69e1-4254-8664-f8ea2a3871ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:00:42.952605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.952574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:42.953843 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.953820    2559 patch_prober.go:28] interesting pod/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 23 23:00:42.953967 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:42.953867    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 23 23:00:43.017380 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.017257    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6d716f-69e1-4254-8664-f8ea2a3871ab-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:00:43.554014 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.553979    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"
Jul 23 23:00:43.554411 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.554013    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9" event={"ID":"bc6d716f-69e1-4254-8664-f8ea2a3871ab","Type":"ContainerDied","Data":"c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1"}
Jul 23 23:00:43.554411 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.554093    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c74da901158a0b50e78cdda53715c69de217930e31b084f70b02c4a761c8a0e1"
Jul 23 23:00:43.953366 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.953260    2559 patch_prober.go:28] interesting pod/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 23 23:00:43.953366 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:43.953334    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 23 23:00:44.952625 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:44.952589    2559 patch_prober.go:28] interesting pod/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 23 23:00:44.953049 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:44.952648    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 23 23:00:45.953203 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:45.953163    2559 patch_prober.go:28] interesting pod/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 23 23:00:45.953664 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:45.953222    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 23 23:00:46.956483 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:46.956453    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:46.956884 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:46.956732    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:00:46.957364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:00:46.957346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:01:00.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:00.192958    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"]
Jul 23 23:01:00.194529 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:00.194506    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-70930e-4gkn9"]
Jul 23 23:01:01.697505 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:01.697471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:01:02.112670 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:02.112634    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc6d716f-69e1-4254-8664-f8ea2a3871ab" path="/var/lib/kubelet/pods/bc6d716f-69e1-4254-8664-f8ea2a3871ab/volumes"
Jul 23 23:01:06.825293 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:06.825255    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:01:06.825779 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:06.825523    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" containerID="cri-o://880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1" gracePeriod=30
Jul 23 23:01:10.583401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:10.583362    2559 patch_prober.go:28] interesting pod/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 23 23:01:10.583794 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:10.583432    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:01:12.073748 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.073721    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:01:12.192008 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.191887    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192008 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.191953    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192008 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.191985    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192066    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6mm4\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192173    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192206    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192224    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192246    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192309    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192624 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192343    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert\") pod \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\" (UID: \"d5e9200b-c144-48bb-b1f1-654aa5bf4edf\") "
Jul 23 23:01:12.192624 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192438    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:12.192624 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:12.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192726    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:01:12.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192834    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192850    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192864    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.192993 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.192877    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.193259 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.193241    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:12.194415 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.194392    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:01:12.194831 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.194815    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:12.194831 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.194819    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4" (OuterVolumeSpecName: "kube-api-access-j6mm4") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "kube-api-access-j6mm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:01:12.195057 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.195034    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5e9200b-c144-48bb-b1f1-654aa5bf4edf" (UID: "d5e9200b-c144-48bb-b1f1-654aa5bf4edf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:01:12.293841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.293786    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.293841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.293833    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.293841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.293845    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.294117 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.293855    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6mm4\" (UniqueName: \"kubernetes.io/projected/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-kube-api-access-j6mm4\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.294117 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.293870    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e9200b-c144-48bb-b1f1-654aa5bf4edf-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:12.691225 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.691186    2559 generic.go:356] "Generic (PLEG): container finished" podID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerID="880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1" exitCode=0
Jul 23 23:01:12.691396 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.691242    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"
Jul 23 23:01:12.691396 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.691267    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerDied","Data":"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"}
Jul 23 23:01:12.691396 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.691323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9" event={"ID":"d5e9200b-c144-48bb-b1f1-654aa5bf4edf","Type":"ContainerDied","Data":"f9c092035504eca17a158b0b8372892e476209e1b9c764e4981005a4869141b6"}
Jul 23 23:01:12.691396 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.691345    2559 scope.go:117] "RemoveContainer" containerID="880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"
Jul 23 23:01:12.700516 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.700498    2559 scope.go:117] "RemoveContainer" containerID="45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"
Jul 23 23:01:12.708069 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.708053    2559 scope.go:117] "RemoveContainer" containerID="880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"
Jul 23 23:01:12.708316 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:01:12.708296    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1\": container with ID starting with 880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1 not found: ID does not exist" containerID="880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"
Jul 23 23:01:12.708363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.708324    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1"} err="failed to get container status \"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1\": rpc error: code = NotFound desc = could not find container \"880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1\": container with ID starting with 880b163af92384ec2462b61c90ed1126b8038ac6b3a043bf22666793033ee2b1 not found: ID does not exist"
Jul 23 23:01:12.708363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.708339    2559 scope.go:117] "RemoveContainer" containerID="45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"
Jul 23 23:01:12.708593 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:01:12.708575    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835\": container with ID starting with 45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835 not found: ID does not exist" containerID="45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"
Jul 23 23:01:12.708642 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.708599    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835"} err="failed to get container status \"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835\": rpc error: code = NotFound desc = could not find container \"45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835\": container with ID starting with 45d41ab716d4c2561bd4cc3b4f4e322d9a0b7a266b47327498484fed47688835 not found: ID does not exist"
Jul 23 23:01:12.713298 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.712426    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:01:12.719117 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:12.719090    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-70930e-openshift-default-7bbb95494c-48xt9"]
Jul 23 23:01:14.110624 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:14.110589    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" path="/var/lib/kubelet/pods/d5e9200b-c144-48bb-b1f1-654aa5bf4edf/volumes"
Jul 23 23:01:26.999600 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:26.999562    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:01:34.030330 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.030296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:01:34.030727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.030576    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy" containerID="cri-o://6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a" gracePeriod=30
Jul 23 23:01:34.982609 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.982577    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:01:34.983013 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.982994    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:34.983103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983016    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:34.983103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983046    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:34.983103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983055    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:34.983103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983073    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6d716f-69e1-4254-8664-f8ea2a3871ab" containerName="revoke-keys"
Jul 23 23:01:34.983103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983079    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6d716f-69e1-4254-8664-f8ea2a3871ab" containerName="revoke-keys"
Jul 23 23:01:34.983371 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983140    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:34.983371 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.983149    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6d716f-69e1-4254-8664-f8ea2a3871ab" containerName="revoke-keys"
Jul 23 23:01:34.985237 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.985220    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:34.988952 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.988930    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-fd1be5-openshift-default-dockercfg-nl2bs\""
Jul 23 23:01:34.997643 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:34.997622    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:01:35.087304 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmps9\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.087727 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.087584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188322 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188285    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmps9\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188460    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188786 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188786 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.188922 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.188837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189216    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.189768 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.189526    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.191096 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.191073    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.191364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.191345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.195625 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.195597    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.195922 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.195888    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmps9\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9\") pod \"e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") " pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.297154 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.297068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:35.431657 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.431582    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:01:35.433861 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:01:35.433834    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6f79fb_b04d_48f5_bf67_7fd8b81e506c.slice/crio-e9da2fe803974ce6c5c4c29cab301422d2cf48f6888298922e24cd6f984f29b8 WatchSource:0}: Error finding container e9da2fe803974ce6c5c4c29cab301422d2cf48f6888298922e24cd6f984f29b8: Status 404 returned error can't find the container with id e9da2fe803974ce6c5c4c29cab301422d2cf48f6888298922e24cd6f984f29b8
Jul 23 23:01:35.436230 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.436187    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:35.436309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.436280    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:35.436349 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.436320    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:35.796533 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.796496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerStarted","Data":"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"}
Jul 23 23:01:35.796712 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.796549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerStarted","Data":"e9da2fe803974ce6c5c4c29cab301422d2cf48f6888298922e24cd6f984f29b8"}
Jul 23 23:01:35.816639 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:35.816586    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podStartSLOduration=1.816570731 podStartE2EDuration="1.816570731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:01:35.814400111 +0000 UTC m=+2326.178066525" watchObservedRunningTime="2026-07-23 23:01:35.816570731 +0000 UTC m=+2326.180237131"
Jul 23 23:01:36.298010 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:36.297969    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:36.302824 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:36.302798    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:36.801580 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:36.801544    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:36.802736 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:36.802716    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:39.287233 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.287208    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:01:39.329142 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329055    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329142 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329142 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329129    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329149    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329172    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n9qf\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329204    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329277    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329306    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data\") pod \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\" (UID: \"92b68eef-2a07-4da6-9fd9-2c4a29f12743\") "
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:39.329431 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329416    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:39.329865 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329663    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.329865 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329683    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.329865 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329715    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:39.329865 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.329740    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:01:39.330685 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.330655    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data" (OuterVolumeSpecName: "istio-data") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:39.331672 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.331647    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:01:39.331790 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.331683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token" (OuterVolumeSpecName: "istio-token") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:01:39.332114 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.332092    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:01:39.332315 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.332295    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf" (OuterVolumeSpecName: "kube-api-access-9n9qf") pod "92b68eef-2a07-4da6-9fd9-2c4a29f12743" (UID: "92b68eef-2a07-4da6-9fd9-2c4a29f12743"). InnerVolumeSpecName "kube-api-access-9n9qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:01:39.430559 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430509    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430559 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430551    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430559 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430563    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430559 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430574    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430586    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n9qf\" (UniqueName: \"kubernetes.io/projected/92b68eef-2a07-4da6-9fd9-2c4a29f12743-kube-api-access-9n9qf\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430597    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.430841 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.430607    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b68eef-2a07-4da6-9fd9-2c4a29f12743-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:01:39.822211 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.822175    2559 generic.go:356] "Generic (PLEG): container finished" podID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerID="6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a" exitCode=0
Jul 23 23:01:39.822364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.822224    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"
Jul 23 23:01:39.822364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.822265    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerDied","Data":"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"}
Jul 23 23:01:39.822364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.822345    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml" event={"ID":"92b68eef-2a07-4da6-9fd9-2c4a29f12743","Type":"ContainerDied","Data":"84ba02d46dbf83351045ea00f8eddf12594daae5f72790e7fc948cbd71e7cdbd"}
Jul 23 23:01:39.822496 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.822368    2559 scope.go:117] "RemoveContainer" containerID="6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"
Jul 23 23:01:39.831604 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.831585    2559 scope.go:117] "RemoveContainer" containerID="bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"
Jul 23 23:01:39.840190 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.840171    2559 scope.go:117] "RemoveContainer" containerID="6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"
Jul 23 23:01:39.840486 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:01:39.840463    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a\": container with ID starting with 6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a not found: ID does not exist" containerID="6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"
Jul 23 23:01:39.840537 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.840498    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a"} err="failed to get container status \"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a\": rpc error: code = NotFound desc = could not find container \"6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a\": container with ID starting with 6f424384702fceb778858b384176c5737e2f422b6bdcc69f533985f18afc9c5a not found: ID does not exist"
Jul 23 23:01:39.840537 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.840517    2559 scope.go:117] "RemoveContainer" containerID="bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"
Jul 23 23:01:39.840769 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:01:39.840751    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b\": container with ID starting with bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b not found: ID does not exist" containerID="bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"
Jul 23 23:01:39.840845 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.840778    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b"} err="failed to get container status \"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b\": rpc error: code = NotFound desc = could not find container \"bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b\": container with ID starting with bc93d8f59e5a1aa33094d789dd7e5255408ddd52d13ffe608adc6d2d67bd749b not found: ID does not exist"
Jul 23 23:01:39.845522 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.845488    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:01:39.850881 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:39.850855    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-857bc8-openshift-default-79c9d57ddc-lhtml"]
Jul 23 23:01:40.110066 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:40.109965    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" path="/var/lib/kubelet/pods/92b68eef-2a07-4da6-9fd9-2c4a29f12743/volumes"
Jul 23 23:01:42.990784 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.990745    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:01:42.991222 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.991206    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:01:42.991269 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.991225    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:01:42.991305 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.991291    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5e9200b-c144-48bb-b1f1-654aa5bf4edf" containerName="istio-proxy"
Jul 23 23:01:42.991305 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.991300    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:01:42.991364 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.991306    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:01:42.994098 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.994080    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:42.996499 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:42.996477    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-159535-openshift-default-dockercfg-dj7b4\""
Jul 23 23:01:43.067536 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067544    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ds4\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067655    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067717 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067976 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.067976 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.067763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.092345 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.092310    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:01:43.169176 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169312    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169383 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169656 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169656 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169656 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ds4\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169656 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169575    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.169955 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.170021 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.169988    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.170196 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.170147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.171666 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.171640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.171882 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.171866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.176357 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.176337    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.176634 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.176615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ds4\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4\") pod \"e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") " pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.304420 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.304328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:43.460119 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.460095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:01:43.462633 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:01:43.462605    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bf818c_09a3_42d1_94c8_e1eae259606a.slice/crio-a1b686240ea1590894c8330bed06a391374f92bfe0fb6b8ad4af66eddac2241a WatchSource:0}: Error finding container a1b686240ea1590894c8330bed06a391374f92bfe0fb6b8ad4af66eddac2241a: Status 404 returned error can't find the container with id a1b686240ea1590894c8330bed06a391374f92bfe0fb6b8ad4af66eddac2241a
Jul 23 23:01:43.464695 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.464658    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:43.464787 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.464730    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:43.464787 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.464763    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:43.846688 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.846650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerStarted","Data":"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"}
Jul 23 23:01:43.846873 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.846736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerStarted","Data":"a1b686240ea1590894c8330bed06a391374f92bfe0fb6b8ad4af66eddac2241a"}
Jul 23 23:01:43.865245 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:43.865187    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podStartSLOduration=1.865144715 podStartE2EDuration="1.865144715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:01:43.864356217 +0000 UTC m=+2334.228022601" watchObservedRunningTime="2026-07-23 23:01:43.865144715 +0000 UTC m=+2334.228811080"
Jul 23 23:01:44.305116 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:44.305073    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:44.310155 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:44.310131    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:44.850878 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:44.850839    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:44.852018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:44.851996    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:01:45.496037 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:45.496003    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:01:52.843157 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:52.843116    2559 patch_prober.go:28] interesting pod/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 23:01:52.843612 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:52.843190    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 23:01:53.909621 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.909581    2559 generic.go:356] "Generic (PLEG): container finished" podID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerID="472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506" exitCode=0
Jul 23 23:01:53.910178 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.909657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerDied","Data":"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"}
Jul 23 23:01:53.910291 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.910267    2559 scope.go:117] "RemoveContainer" containerID="472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"
Jul 23 23:01:53.912643 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.912597    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:53.912905 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.912864    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:53.913291 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:53.913251    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:01:54.919542 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:54.919491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerStarted","Data":"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"}
Jul 23 23:01:55.298131 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:55.298094    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:55.298520 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:55.298422    2559 patch_prober.go:28] interesting pod/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 23:01:55.298684 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:55.298663    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 23:01:56.298381 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:56.298333    2559 patch_prober.go:28] interesting pod/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 23:01:56.298926 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:56.298404    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 23:01:56.494337 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:56.494300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:01:57.298513 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:57.298479    2559 patch_prober.go:28] interesting pod/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 23:01:57.299089 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:57.299055    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 23:01:58.300453 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:58.300407    2559 patch_prober.go:28] interesting pod/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 23 23:01:58.300938 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:58.300479    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 23 23:01:59.301876 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:59.301843    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:59.302310 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:59.302242    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:01:59.303028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:01:59.303007    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:02:00.428338 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.427188    2559 patch_prober.go:28] interesting pod/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:56430->10.133.0.50:15021: read: connection reset by peer" start-of-body=
Jul 23 23:02:00.428338 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.427276    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:56430->10.133.0.50:15021: read: connection reset by peer"
Jul 23 23:02:00.959358 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.959330    2559 generic.go:356] "Generic (PLEG): container finished" podID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerID="fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab" exitCode=0
Jul 23 23:02:00.959530 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.959399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerDied","Data":"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"}
Jul 23 23:02:00.959941 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.959925    2559 scope.go:117] "RemoveContainer" containerID="fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"
Jul 23 23:02:00.960936 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.960881    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:02:00.961002 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.960974    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:02:00.961038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:00.961012    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:02:01.965714 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:01.965675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerStarted","Data":"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"}
Jul 23 23:02:02.304824 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:02.304788    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:02:02.305250 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:02.305150    2559 patch_prober.go:28] interesting pod/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:02:02.305441 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:02.305418    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 23:02:03.305614 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:03.305574    2559 patch_prober.go:28] interesting pod/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:02:03.306234 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:03.306198    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 23:02:04.305843 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:04.305800    2559 patch_prober.go:28] interesting pod/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:02:04.306489 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:04.306458    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 23:02:05.305187 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:05.305144    2559 patch_prober.go:28] interesting pod/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:02:05.305519 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:05.305482    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 23 23:02:06.308928 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:06.308875    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:02:06.309382 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:06.309178    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:02:06.310073 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:06.310052    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:02:07.597273 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.597235    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:02:07.597724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.597708    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:02:07.597769 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.597727    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="92b68eef-2a07-4da6-9fd9-2c4a29f12743" containerName="istio-proxy"
Jul 23 23:02:07.601327 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.601311    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.603794 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.603773    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"auth-test-model-fd1be5-kserve-self-signed-certs\""
Jul 23 23:02:07.604581 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.604563    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"openshift-service-ca.crt\""
Jul 23 23:02:07.604665 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.604588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"default-dockercfg-nr6qn\""
Jul 23 23:02:07.604665 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.604641    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"kube-root-ca.crt\""
Jul 23 23:02:07.609346 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.609319    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:02:07.727586 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2ljz\" (UniqueName: \"kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.727795 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.727795 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.727795 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.727989 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.727989 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.727863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.828951 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.828908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.828951 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.828960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2ljz\" (UniqueName: \"kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829199 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.828989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829199 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829199 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829199 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829451 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829522 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.829586 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.829556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.831388 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.831367    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.831639 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.831623    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.837123 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.837097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2ljz\" (UniqueName: \"kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz\") pod \"auth-test-model-fd1be5-kserve-54b497dff-88sk8\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") " pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:07.911392 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:07.911300    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:08.046937 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:08.046883    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:02:08.051970 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:02:08.051923    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87fcd566_8bb8_40c4_aa16_55fb013c8c68.slice/crio-afd221f12e92aaa5e6d81d231a4dfae98214b8909b5ef15009f8e9ec6460b4fa WatchSource:0}: Error finding container afd221f12e92aaa5e6d81d231a4dfae98214b8909b5ef15009f8e9ec6460b4fa: Status 404 returned error can't find the container with id afd221f12e92aaa5e6d81d231a4dfae98214b8909b5ef15009f8e9ec6460b4fa
Jul 23 23:02:09.000140 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:09.000098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" event={"ID":"87fcd566-8bb8-40c4-aa16-55fb013c8c68","Type":"ContainerStarted","Data":"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"}
Jul 23 23:02:09.000524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:09.000155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" event={"ID":"87fcd566-8bb8-40c4-aa16-55fb013c8c68","Type":"ContainerStarted","Data":"afd221f12e92aaa5e6d81d231a4dfae98214b8909b5ef15009f8e9ec6460b4fa"}
Jul 23 23:02:09.017102 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:09.017052    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" podStartSLOduration=2.017034416 podStartE2EDuration="2.017034416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:02:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:02:09.015064304 +0000 UTC m=+2359.378730710" watchObservedRunningTime="2026-07-23 23:02:09.017034416 +0000 UTC m=+2359.380700798"
Jul 23 23:02:10.995884 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:10.995843    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:02:17.912336 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:17.912274    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:17.912336 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:17.912354    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:17.925268 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:17.925243    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:18.053705 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:18.053671    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:02:27.003656 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.003619    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:02:27.007046 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.007021    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.009441 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.009411    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"openshift-service-ca.crt\""
Jul 23 23:02:27.010288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.010256    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"kube-root-ca.crt\""
Jul 23 23:02:27.010386 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.010326    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"auth-test-model-159535-kserve-self-signed-certs\""
Jul 23 23:02:27.010449 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.010432    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"default-dockercfg-jbptr\""
Jul 23 23:02:27.015878 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.015852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:02:27.093469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.093646 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.093646 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.093646 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.093805 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klm6\" (UniqueName: \"kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.093805 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.093711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194436 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194436 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klm6\" (UniqueName: \"kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194662 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194662 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194662 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194529    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194662 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194888 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.194984 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194953    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.195021 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.194982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.197001 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.196909    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.197202 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.197185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.201786 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.201763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klm6\" (UniqueName: \"kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6\") pod \"auth-test-model-159535-kserve-5b698cb9b7-g7wjz\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") " pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.318060 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.317967    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:27.453239 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:27.453213    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:02:27.455332 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:02:27.455307    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod417a9ee0_0ea5_4ca5_b585_c7581c19fd1b.slice/crio-fe28167c27f69b3cade8ce6b6c30b06021ae53d552679a103a8a300755f7ba19 WatchSource:0}: Error finding container fe28167c27f69b3cade8ce6b6c30b06021ae53d552679a103a8a300755f7ba19: Status 404 returned error can't find the container with id fe28167c27f69b3cade8ce6b6c30b06021ae53d552679a103a8a300755f7ba19
Jul 23 23:02:28.093528 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:28.093486    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" event={"ID":"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b","Type":"ContainerStarted","Data":"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"}
Jul 23 23:02:28.093957 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:28.093543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" event={"ID":"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b","Type":"ContainerStarted","Data":"fe28167c27f69b3cade8ce6b6c30b06021ae53d552679a103a8a300755f7ba19"}
Jul 23 23:02:28.108926 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:28.108858    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" podStartSLOduration=2.108841458 podStartE2EDuration="2.108841458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:02:28.107557884 +0000 UTC m=+2378.471224268" watchObservedRunningTime="2026-07-23 23:02:28.108841458 +0000 UTC m=+2378.472507882"
Jul 23 23:02:30.201039 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:30.200998    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:02:37.318203 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:37.318165    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:37.318203 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:37.318210    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:37.330760 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:37.330734    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:38.154006 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:38.153974    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:02:49.296081 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:02:49.296048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:03:04.397669 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:03:04.397624    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:03:22.901379 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:03:22.901299    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:03:38.596984 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:03:38.596944    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:03:56.394247 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:03:56.394211    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:04:12.496556 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:12.496519    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:04:26.843150 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:26.843107    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:04:27.797836 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:27.797806    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:04:29.293054 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:29.293017    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:04:41.692622 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:41.692538    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:04:41.864115 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:41.864074    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" podUID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" containerName="main" containerID="cri-o://f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0" gracePeriod=45
Jul 23 23:04:42.120667 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.120641    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:04:42.232699 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232601    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.232699 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232659    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2ljz\" (UniqueName: \"kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.232699 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.232976 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232715    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.232976 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232754    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.232976 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232849    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir\") pod \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\" (UID: \"87fcd566-8bb8-40c4-aa16-55fb013c8c68\") "
Jul 23 23:04:42.233104 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.232979    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home" (OuterVolumeSpecName: "home") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:42.233104 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.233047    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache" (OuterVolumeSpecName: "model-cache") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:42.233194 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.233173    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:42.233281 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.233259    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.233336 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.233286    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.233336 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.233299    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.235270 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.235242    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz" (OuterVolumeSpecName: "kube-api-access-w2ljz") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "kube-api-access-w2ljz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:04:42.235400 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.235266    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm" (OuterVolumeSpecName: "dshm") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:42.235400 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.235272    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87fcd566-8bb8-40c4-aa16-55fb013c8c68" (UID: "87fcd566-8bb8-40c4-aa16-55fb013c8c68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:04:42.334711 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.334672    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2ljz\" (UniqueName: \"kubernetes.io/projected/87fcd566-8bb8-40c4-aa16-55fb013c8c68-kube-api-access-w2ljz\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.334711 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.334705    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fcd566-8bb8-40c4-aa16-55fb013c8c68-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.334711 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.334714    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fcd566-8bb8-40c4-aa16-55fb013c8c68-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:42.756250 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.756216    2559 generic.go:356] "Generic (PLEG): container finished" podID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" containerID="f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0" exitCode=0
Jul 23 23:04:42.756660 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.756259    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"
Jul 23 23:04:42.756660 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.756302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" event={"ID":"87fcd566-8bb8-40c4-aa16-55fb013c8c68","Type":"ContainerDied","Data":"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"}
Jul 23 23:04:42.756660 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.756350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8" event={"ID":"87fcd566-8bb8-40c4-aa16-55fb013c8c68","Type":"ContainerDied","Data":"afd221f12e92aaa5e6d81d231a4dfae98214b8909b5ef15009f8e9ec6460b4fa"}
Jul 23 23:04:42.756660 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.756370    2559 scope.go:117] "RemoveContainer" containerID="f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"
Jul 23 23:04:42.765874 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.765853    2559 scope.go:117] "RemoveContainer" containerID="f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"
Jul 23 23:04:42.766165 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:04:42.766148    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0\": container with ID starting with f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0 not found: ID does not exist" containerID="f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"
Jul 23 23:04:42.766219 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.766176    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0"} err="failed to get container status \"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0\": rpc error: code = NotFound desc = could not find container \"f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0\": container with ID starting with f72f1100d4fe12b040368a59e6ffa57b724de3585755696887bf5faaba9406b0 not found: ID does not exist"
Jul 23 23:04:42.776252 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.776230    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:04:42.778545 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.778521    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/auth-test-model-fd1be5-kserve-54b497dff-88sk8"]
Jul 23 23:04:42.818671 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:42.818607    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" podUID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" containerName="main" containerID="cri-o://ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71" gracePeriod=45
Jul 23 23:04:43.081545 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.081520    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:04:43.141910 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.141857    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142069 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.141942    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142111 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142068    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142175 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142161    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6klm6\" (UniqueName: \"kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142213 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142186    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142251 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142214    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir\") pod \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\" (UID: \"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b\") "
Jul 23 23:04:43.142251 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home" (OuterVolumeSpecName: "home") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:43.142476 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142450    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:43.142788 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142492    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:43.142788 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142530    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.142788 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.142548    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.144390 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.144364    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm" (OuterVolumeSpecName: "dshm") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:04:43.144390 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.144370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6" (OuterVolumeSpecName: "kube-api-access-6klm6") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "kube-api-access-6klm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:04:43.144522 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.144400    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" (UID: "417a9ee0-0ea5-4ca5-b585-c7581c19fd1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:04:43.243068 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.243026    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.243068 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.243059    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.243068 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.243071    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6klm6\" (UniqueName: \"kubernetes.io/projected/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-kube-api-access-6klm6\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.243311 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.243080    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:04:43.762518 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.762485    2559 generic.go:356] "Generic (PLEG): container finished" podID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" containerID="ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71" exitCode=0
Jul 23 23:04:43.763032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.762546    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"
Jul 23 23:04:43.763032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.762575    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" event={"ID":"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b","Type":"ContainerDied","Data":"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"}
Jul 23 23:04:43.763032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.762621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz" event={"ID":"417a9ee0-0ea5-4ca5-b585-c7581c19fd1b","Type":"ContainerDied","Data":"fe28167c27f69b3cade8ce6b6c30b06021ae53d552679a103a8a300755f7ba19"}
Jul 23 23:04:43.763032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.762646    2559 scope.go:117] "RemoveContainer" containerID="ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"
Jul 23 23:04:43.772117 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.772092    2559 scope.go:117] "RemoveContainer" containerID="ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"
Jul 23 23:04:43.772388 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:04:43.772368    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71\": container with ID starting with ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71 not found: ID does not exist" containerID="ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"
Jul 23 23:04:43.772439 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.772398    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71"} err="failed to get container status \"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71\": rpc error: code = NotFound desc = could not find container \"ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71\": container with ID starting with ea7626c8201b42ca6451271af8a6833f7e514c864af55922b47c20b55e3afd71 not found: ID does not exist"
Jul 23 23:04:43.782919 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.782877    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:04:43.786454 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:43.786430    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-159535/auth-test-model-159535-kserve-5b698cb9b7-g7wjz"]
Jul 23 23:04:44.110766 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:44.110682    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" path="/var/lib/kubelet/pods/417a9ee0-0ea5-4ca5-b585-c7581c19fd1b/volumes"
Jul 23 23:04:44.111151 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:44.111136    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" path="/var/lib/kubelet/pods/87fcd566-8bb8-40c4-aa16-55fb013c8c68/volumes"
Jul 23 23:04:51.806203 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806170    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:04:51.806565 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806500    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" containerName="main"
Jul 23 23:04:51.806565 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806511    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" containerName="main"
Jul 23 23:04:51.806565 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806529    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" containerName="main"
Jul 23 23:04:51.806565 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806535    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" containerName="main"
Jul 23 23:04:51.806696 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806589    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="417a9ee0-0ea5-4ca5-b585-c7581c19fd1b" containerName="main"
Jul 23 23:04:51.806696 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.806611    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="87fcd566-8bb8-40c4-aa16-55fb013c8c68" containerName="main"
Jul 23 23:04:51.810834 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.810814    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.814053 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.814023    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"default-dockercfg-nr6qn\""
Jul 23 23:04:51.814208 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.814035    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"kube-root-ca.crt\""
Jul 23 23:04:51.814208 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.814061    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"sub-test-model-fd1be5-kserve-self-signed-certs\""
Jul 23 23:04:51.814208 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.814063    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"openshift-service-ca.crt\""
Jul 23 23:04:51.819172 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.819148    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:04:51.924747 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.924711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.924747 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.924761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.925032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.924840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.925032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.924865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.925032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.924889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:51.925032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:51.925003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4qpz\" (UniqueName: \"kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026266 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026266 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026502 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026502 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4qpz\" (UniqueName: \"kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026502 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026502 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026763 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026880 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.026880 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.026822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.028803 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.028783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.029226 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.029205    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.033929 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.033888    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4qpz\" (UniqueName: \"kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz\") pod \"sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") " pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.121303 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.121215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:04:52.255849 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.255826    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:04:52.258385 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:04:52.258351    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90473aed_31a0_472c_a090_0b0eb0a5bed3.slice/crio-535511b8fec61d1f2580e9f2f0c429628fc6a22a168464bca6c8ee1315963588 WatchSource:0}: Error finding container 535511b8fec61d1f2580e9f2f0c429628fc6a22a168464bca6c8ee1315963588: Status 404 returned error can't find the container with id 535511b8fec61d1f2580e9f2f0c429628fc6a22a168464bca6c8ee1315963588
Jul 23 23:04:52.809831 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.809796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" event={"ID":"90473aed-31a0-472c-a090-0b0eb0a5bed3","Type":"ContainerStarted","Data":"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"}
Jul 23 23:04:52.809831 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.809837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" event={"ID":"90473aed-31a0-472c-a090-0b0eb0a5bed3","Type":"ContainerStarted","Data":"535511b8fec61d1f2580e9f2f0c429628fc6a22a168464bca6c8ee1315963588"}
Jul 23 23:04:52.826469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:52.826390    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" podStartSLOduration=1.826374528 podStartE2EDuration="1.826374528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:04:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:04:52.826095885 +0000 UTC m=+2523.189762265" watchObservedRunningTime="2026-07-23 23:04:52.826374528 +0000 UTC m=+2523.190040916"
Jul 23 23:04:55.302235 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:04:55.302192    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:05:02.122201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:02.122163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:05:02.122810 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:02.122221    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:05:02.134734 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:02.134705    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:05:02.866668 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:02.866642    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:05:15.007014 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.006976    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:05:15.010503 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.010480    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.013823 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.013798    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"openshift-service-ca.crt\""
Jul 23 23:05:15.013959 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.013825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"sub-test-model-159535-kserve-self-signed-certs\""
Jul 23 23:05:15.013959 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.013798    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"kube-root-ca.crt\""
Jul 23 23:05:15.013959 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.013914    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"default-dockercfg-jbptr\""
Jul 23 23:05:15.017324 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.017300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:05:15.147854 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.147818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.148077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.147864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m4fh\" (UniqueName: \"kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.148077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.147889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.148077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.147963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.148077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.147993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.148077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.148067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.248875 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.248831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249102 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.248931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249102 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.248974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249102 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249281 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m4fh\" (UniqueName: \"kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249281 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249432 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249513 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249460    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.249564 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.249506    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.251318 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.251301    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.251624 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.251609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.256549 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.256524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m4fh\" (UniqueName: \"kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh\") pod \"sub-test-model-159535-kserve-6b7d487587-44pww\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") " pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.321517 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.321434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:15.457412 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.457389    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:05:15.459439 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:05:15.459406    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c1010a9_ee11_4a07_a0c1_dd39e9fbc632.slice/crio-2fdf471f827eb4fd3eb61aa22cf3be0bfe214a24c82cfe8cd513a3c17e28f2b3 WatchSource:0}: Error finding container 2fdf471f827eb4fd3eb61aa22cf3be0bfe214a24c82cfe8cd513a3c17e28f2b3: Status 404 returned error can't find the container with id 2fdf471f827eb4fd3eb61aa22cf3be0bfe214a24c82cfe8cd513a3c17e28f2b3
Jul 23 23:05:15.916343 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.916307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" event={"ID":"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632","Type":"ContainerStarted","Data":"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"}
Jul 23 23:05:15.916524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.916353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" event={"ID":"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632","Type":"ContainerStarted","Data":"2fdf471f827eb4fd3eb61aa22cf3be0bfe214a24c82cfe8cd513a3c17e28f2b3"}
Jul 23 23:05:15.933609 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:15.933562    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" podStartSLOduration=1.933546854 podStartE2EDuration="1.933546854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:05:15.932936507 +0000 UTC m=+2546.296602888" watchObservedRunningTime="2026-07-23 23:05:15.933546854 +0000 UTC m=+2546.297213278"
Jul 23 23:05:18.499591 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:18.499552    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:05:25.322549 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:25.322504    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:25.323018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:25.322587    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:25.334695 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:25.334669    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:25.977603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:25.977574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:05:36.191699 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:36.191655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:05:56.995493 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:05:56.995453    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:06:02.781996 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:02.781960    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:06:03.678229 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:03.678196    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:06:07.103499 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.103460    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:06:07.108918 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.108864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.113868 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.113840    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd1be5\"/\"rate-test-model-fd1be5-kserve-self-signed-certs\""
Jul 23 23:06:07.116079 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.116055    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:06:07.161879 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.161835    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.162080 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.161952    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.162080 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.162001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.162080 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.162026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.162080 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.162065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.162237 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.162115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g29hw\" (UniqueName: \"kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.263695 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.263662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.263876 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.263718    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.263959 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.263870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.263959 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.263942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.264049 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.263971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.264049 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.264021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g29hw\" (UniqueName: \"kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.264178 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.264157    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.264249 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.264231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.264347 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.264322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.266373 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.266347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.266631 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.266612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.272060 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.272038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g29hw\" (UniqueName: \"kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw\") pod \"rate-test-model-fd1be5-kserve-5584cfc979-bggzb\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") " pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.420955 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.420842    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:07.557304 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:07.557278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:06:07.559441 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:06:07.559406    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3d3b98_2a7d_4271_880b_f1d8f6317aca.slice/crio-42fbe14664c33c61556b6f3e5e7817ba73a13b7f0a74079aae3011b27518148a WatchSource:0}: Error finding container 42fbe14664c33c61556b6f3e5e7817ba73a13b7f0a74079aae3011b27518148a: Status 404 returned error can't find the container with id 42fbe14664c33c61556b6f3e5e7817ba73a13b7f0a74079aae3011b27518148a
Jul 23 23:06:08.180334 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:08.180295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" event={"ID":"ed3d3b98-2a7d-4271-880b-f1d8f6317aca","Type":"ContainerStarted","Data":"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"}
Jul 23 23:06:08.180334 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:08.180339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" event={"ID":"ed3d3b98-2a7d-4271-880b-f1d8f6317aca","Type":"ContainerStarted","Data":"42fbe14664c33c61556b6f3e5e7817ba73a13b7f0a74079aae3011b27518148a"}
Jul 23 23:06:08.197199 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:08.197148    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" podStartSLOduration=1.197132095 podStartE2EDuration="1.197132095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:06:08.195118954 +0000 UTC m=+2598.558785339" watchObservedRunningTime="2026-07-23 23:06:08.197132095 +0000 UTC m=+2598.560798477"
Jul 23 23:06:15.999026 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:15.998939    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:06:17.421910 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:17.421853    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:17.422368 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:17.421935    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:17.434515 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:17.434483    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:17.803881 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:17.803812    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" podUID="90473aed-31a0-472c-a090-0b0eb0a5bed3" containerName="main" containerID="cri-o://93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2" gracePeriod=45
Jul 23 23:06:18.054243 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.054178    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:06:18.173946 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.173883    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174139 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.173985    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174139 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174041    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4qpz\" (UniqueName: \"kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174139 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174084    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174139 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174114    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174350 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174163    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir\") pod \"90473aed-31a0-472c-a090-0b0eb0a5bed3\" (UID: \"90473aed-31a0-472c-a090-0b0eb0a5bed3\") "
Jul 23 23:06:18.174350 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174165    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home" (OuterVolumeSpecName: "home") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:18.174447 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174414    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache" (OuterVolumeSpecName: "model-cache") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:18.174447 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174432    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:18.174546 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174532    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.174604 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174550    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.174604 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.174582    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.176229 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.176207    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm" (OuterVolumeSpecName: "dshm") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:18.176360 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.176284    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:06:18.176360 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.176304    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz" (OuterVolumeSpecName: "kube-api-access-q4qpz") pod "90473aed-31a0-472c-a090-0b0eb0a5bed3" (UID: "90473aed-31a0-472c-a090-0b0eb0a5bed3"). InnerVolumeSpecName "kube-api-access-q4qpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:06:18.230156 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.230125    2559 generic.go:356] "Generic (PLEG): container finished" podID="90473aed-31a0-472c-a090-0b0eb0a5bed3" containerID="93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2" exitCode=0
Jul 23 23:06:18.230321 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.230168    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"
Jul 23 23:06:18.230321 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.230210    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" event={"ID":"90473aed-31a0-472c-a090-0b0eb0a5bed3","Type":"ContainerDied","Data":"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"}
Jul 23 23:06:18.230321 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.230253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9" event={"ID":"90473aed-31a0-472c-a090-0b0eb0a5bed3","Type":"ContainerDied","Data":"535511b8fec61d1f2580e9f2f0c429628fc6a22a168464bca6c8ee1315963588"}
Jul 23 23:06:18.230321 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.230270    2559 scope.go:117] "RemoveContainer" containerID="93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"
Jul 23 23:06:18.239607 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.239592    2559 scope.go:117] "RemoveContainer" containerID="93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"
Jul 23 23:06:18.239880 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:06:18.239859    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2\": container with ID starting with 93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2 not found: ID does not exist" containerID="93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"
Jul 23 23:06:18.239956 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.239911    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2"} err="failed to get container status \"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2\": rpc error: code = NotFound desc = could not find container \"93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2\": container with ID starting with 93e1eada7a8cbc812e077fb6ec38d0ee158509d0e64bba850c5fc73df7a17ca2 not found: ID does not exist"
Jul 23 23:06:18.242671 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.242643    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:06:18.250823 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.250801    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:06:18.254724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.254704    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/sub-test-model-fd1be5-kserve-585cd68bd9-mpcf9"]
Jul 23 23:06:18.275971 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.275941    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90473aed-31a0-472c-a090-0b0eb0a5bed3-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.275971 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.275966    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4qpz\" (UniqueName: \"kubernetes.io/projected/90473aed-31a0-472c-a090-0b0eb0a5bed3-kube-api-access-q4qpz\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.276159 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.275977    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90473aed-31a0-472c-a090-0b0eb0a5bed3-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:18.699477 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.699410    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" podUID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" containerName="main" containerID="cri-o://654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3" gracePeriod=45
Jul 23 23:06:18.953680 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:18.953623    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:06:19.084908 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.084865    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085073 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.084961    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085073 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.084981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085073 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085040    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085226 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085106    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085226 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m4fh\" (UniqueName: \"kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh\") pod \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\" (UID: \"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632\") "
Jul 23 23:06:19.085335 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:19.085335 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085239    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home" (OuterVolumeSpecName: "home") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:19.085450 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085378    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:19.085497 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085452    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.085497 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.085474    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.087163 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.087141    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm" (OuterVolumeSpecName: "dshm") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:06:19.087345 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.087319    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh" (OuterVolumeSpecName: "kube-api-access-6m4fh") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "kube-api-access-6m4fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:06:19.087447 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.087347    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" (UID: "7c1010a9-ee11-4a07-a0c1-dd39e9fbc632"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:06:19.186859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.186819    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.186859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.186851    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.186859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.186860    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m4fh\" (UniqueName: \"kubernetes.io/projected/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-kube-api-access-6m4fh\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.186859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.186869    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:06:19.238347 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.238312    2559 generic.go:356] "Generic (PLEG): container finished" podID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" containerID="654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3" exitCode=0
Jul 23 23:06:19.238511 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.238353    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"
Jul 23 23:06:19.238511 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.238396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" event={"ID":"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632","Type":"ContainerDied","Data":"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"}
Jul 23 23:06:19.238511 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.238436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww" event={"ID":"7c1010a9-ee11-4a07-a0c1-dd39e9fbc632","Type":"ContainerDied","Data":"2fdf471f827eb4fd3eb61aa22cf3be0bfe214a24c82cfe8cd513a3c17e28f2b3"}
Jul 23 23:06:19.238511 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.238455    2559 scope.go:117] "RemoveContainer" containerID="654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"
Jul 23 23:06:19.248222 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.248199    2559 scope.go:117] "RemoveContainer" containerID="654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"
Jul 23 23:06:19.248515 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:06:19.248490    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3\": container with ID starting with 654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3 not found: ID does not exist" containerID="654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"
Jul 23 23:06:19.248596 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.248521    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3"} err="failed to get container status \"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3\": rpc error: code = NotFound desc = could not find container \"654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3\": container with ID starting with 654a5a0e56c2f69c3a42305911731c610280f1250d439bd977ebd6a7b9e1a3f3 not found: ID does not exist"
Jul 23 23:06:19.259871 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.259842    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:06:19.263057 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:19.263033    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-159535/sub-test-model-159535-kserve-6b7d487587-44pww"]
Jul 23 23:06:20.110763 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:20.110735    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" path="/var/lib/kubelet/pods/7c1010a9-ee11-4a07-a0c1-dd39e9fbc632/volumes"
Jul 23 23:06:20.111219 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:20.111203    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90473aed-31a0-472c-a090-0b0eb0a5bed3" path="/var/lib/kubelet/pods/90473aed-31a0-472c-a090-0b0eb0a5bed3/volumes"
Jul 23 23:06:33.904686 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.904654    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905049    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" containerName="main"
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905061    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" containerName="main"
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905076    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90473aed-31a0-472c-a090-0b0eb0a5bed3" containerName="main"
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905080    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="90473aed-31a0-472c-a090-0b0eb0a5bed3" containerName="main"
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905143    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="90473aed-31a0-472c-a090-0b0eb0a5bed3" containerName="main"
Jul 23 23:06:33.905166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.905152    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c1010a9-ee11-4a07-a0c1-dd39e9fbc632" containerName="main"
Jul 23 23:06:33.908114 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.908093    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.910376 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.910352    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"kube-root-ca.crt\""
Jul 23 23:06:33.911232 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.911214    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"rate-test-model-159535-kserve-self-signed-certs\""
Jul 23 23:06:33.911394 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.911375    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"default-dockercfg-jbptr\""
Jul 23 23:06:33.911479 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.911429    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-159535\"/\"openshift-service-ca.crt\""
Jul 23 23:06:33.915671 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.915646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:06:33.927996 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.927961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.928147 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.928002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.928147 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.928035    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.928147 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.928053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdfv6\" (UniqueName: \"kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.928147 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.928079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:33.928363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:33.928229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029589 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029553    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029589 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029832 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029615    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029832 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdfv6\" (UniqueName: \"kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029832 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.029832 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.029731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.030099 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.030045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.030099 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.030086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.030176 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.030127    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.032158 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.032138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.032288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.032270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.036878 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.036858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdfv6\" (UniqueName: \"kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6\") pod \"rate-test-model-159535-kserve-78dd47ddc5-w2h4l\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") " pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.219141 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.219113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:34.352650 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:34.352627    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:06:34.354708 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:06:34.354678    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4517cd04_fcbb_4e07_b353_784bcc72a27b.slice/crio-b611ca15b2504d8973502e54d7fe0c9b0e5de25c69acec26ffb6852fc79df0ff WatchSource:0}: Error finding container b611ca15b2504d8973502e54d7fe0c9b0e5de25c69acec26ffb6852fc79df0ff: Status 404 returned error can't find the container with id b611ca15b2504d8973502e54d7fe0c9b0e5de25c69acec26ffb6852fc79df0ff
Jul 23 23:06:35.315488 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:35.315452    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" event={"ID":"4517cd04-fcbb-4e07-b353-784bcc72a27b","Type":"ContainerStarted","Data":"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"}
Jul 23 23:06:35.315888 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:35.315496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" event={"ID":"4517cd04-fcbb-4e07-b353-784bcc72a27b","Type":"ContainerStarted","Data":"b611ca15b2504d8973502e54d7fe0c9b0e5de25c69acec26ffb6852fc79df0ff"}
Jul 23 23:06:35.333990 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:35.333940    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" podStartSLOduration=2.333923468 podStartE2EDuration="2.333923468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:06:35.331538605 +0000 UTC m=+2625.695204985" watchObservedRunningTime="2026-07-23 23:06:35.333923468 +0000 UTC m=+2625.697589847"
Jul 23 23:06:37.704206 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:37.704162    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:06:44.219795 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:44.219751    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:44.220411 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:44.219807    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:44.232264 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:44.232232    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:44.373855 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:44.373824    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:06:50.370014 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:50.369983    2559 scope.go:117] "RemoveContainer" containerID="8f5fb81d6656c6a4dd025037c7528a3638e3768acf6006b132c3ce7d94e2dcb2"
Jul 23 23:06:50.378289 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:50.378268    2559 scope.go:117] "RemoveContainer" containerID="f072362f9d400b2d8728c8a7e10a2facd7b6988823903923090753a04c8eb561"
Jul 23 23:06:55.000764 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:06:55.000729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:07:10.701416 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:10.701369    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:07:17.646347 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:17.646306    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:07:18.482556 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:18.482520    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:07:25.198965 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:25.198923    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:07:32.667439 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:32.667382    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" podUID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" containerName="main" containerID="cri-o://d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8" gracePeriod=45
Jul 23 23:07:32.920678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:32.920609    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:07:33.003810 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.003774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.003821    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g29hw\" (UniqueName: \"kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.003868    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.003916    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.003963    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004018 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004015    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir\") pod \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\" (UID: \"ed3d3b98-2a7d-4271-880b-f1d8f6317aca\") "
Jul 23 23:07:33.004287 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004126    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.004348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home" (OuterVolumeSpecName: "home") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.004348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004326    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.004469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004450    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.004512 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004477    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.004512 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.004492    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.006222 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.006189    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm" (OuterVolumeSpecName: "dshm") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.006334 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.006262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw" (OuterVolumeSpecName: "kube-api-access-g29hw") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "kube-api-access-g29hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:07:33.006334 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.006308    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed3d3b98-2a7d-4271-880b-f1d8f6317aca" (UID: "ed3d3b98-2a7d-4271-880b-f1d8f6317aca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:07:33.104986 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.104936    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.104986 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.104978    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.104986 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.104988    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g29hw\" (UniqueName: \"kubernetes.io/projected/ed3d3b98-2a7d-4271-880b-f1d8f6317aca-kube-api-access-g29hw\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:33.502979 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.502933    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" podUID="4517cd04-fcbb-4e07-b353-784bcc72a27b" containerName="main" containerID="cri-o://496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309" gracePeriod=45
Jul 23 23:07:33.595909 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.595860    2559 generic.go:356] "Generic (PLEG): container finished" podID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" containerID="d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8" exitCode=0
Jul 23 23:07:33.596090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.595917    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" event={"ID":"ed3d3b98-2a7d-4271-880b-f1d8f6317aca","Type":"ContainerDied","Data":"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"}
Jul 23 23:07:33.596090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.595926    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"
Jul 23 23:07:33.596090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.595950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb" event={"ID":"ed3d3b98-2a7d-4271-880b-f1d8f6317aca","Type":"ContainerDied","Data":"42fbe14664c33c61556b6f3e5e7817ba73a13b7f0a74079aae3011b27518148a"}
Jul 23 23:07:33.596090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.595967    2559 scope.go:117] "RemoveContainer" containerID="d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"
Jul 23 23:07:33.614388 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.614365    2559 scope.go:117] "RemoveContainer" containerID="d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"
Jul 23 23:07:33.614724 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:07:33.614671    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8\": container with ID starting with d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8 not found: ID does not exist" containerID="d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"
Jul 23 23:07:33.614724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.614705    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8"} err="failed to get container status \"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8\": rpc error: code = NotFound desc = could not find container \"d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8\": container with ID starting with d52948aac4c8e3131a9fbee72f145d1193ee141a4891856feaf22795577d16d8 not found: ID does not exist"
Jul 23 23:07:33.615077 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.615053    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:07:33.617510 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.617487    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fd1be5/rate-test-model-fd1be5-kserve-5584cfc979-bggzb"]
Jul 23 23:07:33.763302 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.763239    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:07:33.911978 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.911945    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdfv6\" (UniqueName: \"kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912146 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912146 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912052    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912146 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912099    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912146 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912138    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912379 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs\") pod \"4517cd04-fcbb-4e07-b353-784bcc72a27b\" (UID: \"4517cd04-fcbb-4e07-b353-784bcc72a27b\") "
Jul 23 23:07:33.912448 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912397    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.912448 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912407    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.912448 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.912434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home" (OuterVolumeSpecName: "home") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.914496 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.914461    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm" (OuterVolumeSpecName: "dshm") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:07:33.914496 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.914482    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6" (OuterVolumeSpecName: "kube-api-access-tdfv6") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "kube-api-access-tdfv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:07:33.914646 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:33.914531    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4517cd04-fcbb-4e07-b353-784bcc72a27b" (UID: "4517cd04-fcbb-4e07-b353-784bcc72a27b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013497    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013536    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013549    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4517cd04-fcbb-4e07-b353-784bcc72a27b-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013562    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdfv6\" (UniqueName: \"kubernetes.io/projected/4517cd04-fcbb-4e07-b353-784bcc72a27b-kube-api-access-tdfv6\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013577    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.013601 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.013591    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4517cd04-fcbb-4e07-b353-784bcc72a27b-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:07:34.110797 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.110760    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" path="/var/lib/kubelet/pods/ed3d3b98-2a7d-4271-880b-f1d8f6317aca/volumes"
Jul 23 23:07:34.603156 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.603123    2559 generic.go:356] "Generic (PLEG): container finished" podID="4517cd04-fcbb-4e07-b353-784bcc72a27b" containerID="496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309" exitCode=0
Jul 23 23:07:34.603340 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.603187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" event={"ID":"4517cd04-fcbb-4e07-b353-784bcc72a27b","Type":"ContainerDied","Data":"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"}
Jul 23 23:07:34.603340 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.603203    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"
Jul 23 23:07:34.603340 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.603219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l" event={"ID":"4517cd04-fcbb-4e07-b353-784bcc72a27b","Type":"ContainerDied","Data":"b611ca15b2504d8973502e54d7fe0c9b0e5de25c69acec26ffb6852fc79df0ff"}
Jul 23 23:07:34.603340 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.603235    2559 scope.go:117] "RemoveContainer" containerID="496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"
Jul 23 23:07:34.612142 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.612122    2559 scope.go:117] "RemoveContainer" containerID="496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"
Jul 23 23:07:34.612449 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:07:34.612427    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309\": container with ID starting with 496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309 not found: ID does not exist" containerID="496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"
Jul 23 23:07:34.612510 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.612456    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309"} err="failed to get container status \"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309\": rpc error: code = NotFound desc = could not find container \"496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309\": container with ID starting with 496252e67d0429507af0abfcdeff356b54214a479bb6fb383795586548173309 not found: ID does not exist"
Jul 23 23:07:34.621214 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.621193    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:07:34.627103 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:34.627076    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-159535/rate-test-model-159535-kserve-78dd47ddc5-w2h4l"]
Jul 23 23:07:36.110171 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:36.110139    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4517cd04-fcbb-4e07-b353-784bcc72a27b" path="/var/lib/kubelet/pods/4517cd04-fcbb-4e07-b353-784bcc72a27b/volumes"
Jul 23 23:07:38.475573 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.475541    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:07:38.476028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.475927    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4517cd04-fcbb-4e07-b353-784bcc72a27b" containerName="main"
Jul 23 23:07:38.476028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.475939    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4517cd04-fcbb-4e07-b353-784bcc72a27b" containerName="main"
Jul 23 23:07:38.476028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.475973    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" containerName="main"
Jul 23 23:07:38.476028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.475979    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" containerName="main"
Jul 23 23:07:38.476164 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.476056    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4517cd04-fcbb-4e07-b353-784bcc72a27b" containerName="main"
Jul 23 23:07:38.476164 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.476066    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3d3b98-2a7d-4271-880b-f1d8f6317aca" containerName="main"
Jul 23 23:07:38.480182 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.480165    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.482439 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.482414    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4b3ef9-openshift-default-dockercfg-hgxlx\""
Jul 23 23:07:38.490501 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.490477    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:07:38.560660 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560694    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.560859 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.560855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.561202 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.561017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx7rt\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.561202 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.561069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662114 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662306 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662306 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662306 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662629 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx7rt\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662629 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662629 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662539    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662629 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.662835 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.662632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.663148 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.663128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.663230 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.663148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.665176 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.665121    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.665532 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.665455    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.670516 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.670468    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.670630 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.670550    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx7rt\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt\") pod \"e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") " pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.792005 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.791884    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:38.928726 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.928690    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:07:38.930849 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:07:38.930820    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620508f3_cb97_42de_be0d_7df9dd1fea32.slice/crio-85b76c38fb26f2a936e7b7c08021a9935cbed7ec3f8bd64f70a51404021097cf WatchSource:0}: Error finding container 85b76c38fb26f2a936e7b7c08021a9935cbed7ec3f8bd64f70a51404021097cf: Status 404 returned error can't find the container with id 85b76c38fb26f2a936e7b7c08021a9935cbed7ec3f8bd64f70a51404021097cf
Jul 23 23:07:38.932991 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.932955    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:38.933098 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.933059    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:38.933168 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:38.933112    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:39.629971 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:39.629931    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerStarted","Data":"3ab0311698df1fc3fc5dfd8593cf3a80498dcf5e22c93ca1e851a7bc9c75fe8b"}
Jul 23 23:07:39.630394 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:39.629990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerStarted","Data":"85b76c38fb26f2a936e7b7c08021a9935cbed7ec3f8bd64f70a51404021097cf"}
Jul 23 23:07:39.650804 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:39.650739    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podStartSLOduration=1.650720397 podStartE2EDuration="1.650720397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:07:39.649274719 +0000 UTC m=+2690.012941103" watchObservedRunningTime="2026-07-23 23:07:39.650720397 +0000 UTC m=+2690.014386779"
Jul 23 23:07:39.792412 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:39.792360    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:39.797401 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:39.797376    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:40.634918 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:40.634802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:40.635787 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:40.635767    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:49.991938 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:49.991881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:07:56.676168 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:56.676117    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 23:07:56.676543 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:56.676198    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 23:07:57.725627 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.725545    2559 generic.go:356] "Generic (PLEG): container finished" podID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerID="3ab0311698df1fc3fc5dfd8593cf3a80498dcf5e22c93ca1e851a7bc9c75fe8b" exitCode=0
Jul 23 23:07:57.725627 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.725609    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerDied","Data":"3ab0311698df1fc3fc5dfd8593cf3a80498dcf5e22c93ca1e851a7bc9c75fe8b"}
Jul 23 23:07:57.726082 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.726073    2559 scope.go:117] "RemoveContainer" containerID="3ab0311698df1fc3fc5dfd8593cf3a80498dcf5e22c93ca1e851a7bc9c75fe8b"
Jul 23 23:07:57.727175 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.727147    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:57.727242 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.727229    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:57.727295 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:57.727263    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 23 23:07:58.732612 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:58.732578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerStarted","Data":"02bb821ca8c5abb9f376daf45a8de8c2882bb3b1a97f723242dc8fa811b8e7b3"}
Jul 23 23:07:58.792724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:58.792691    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:07:58.793003 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:58.792974    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 23 23:07:58.793233 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:58.793206    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 23 23:07:59.792814 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:59.792776    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 23 23:07:59.793321 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:07:59.792854    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 23 23:08:00.792549 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:00.792513    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 23 23:08:00.792735 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:00.792567    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 23 23:08:01.793068 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:01.793030    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 23 23:08:01.793703 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:01.793671    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 23 23:08:02.796429 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:02.796395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:08:02.796876 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:02.796669    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:08:02.797467 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:02.797452    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:08:08.196191 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:08.196151    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:08:35.979168 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.979126    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:08:35.982450 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.982429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:35.986027 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.986002    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4b3ef9\"/\"kube-root-ca.crt\""
Jul 23 23:08:35.986027 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.986018    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4b3ef9\"/\"default-dockercfg-nnfzz\""
Jul 23 23:08:35.986201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.986009    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4b3ef9\"/\"ipp-route-4b3ef9-kserve-self-signed-certs\""
Jul 23 23:08:35.986201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.986002    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4b3ef9\"/\"openshift-service-ca.crt\""
Jul 23 23:08:35.991166 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:35.991130    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:08:36.035350 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035319    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.035350 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.035529 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.035529 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b97bf\" (UniqueName: \"kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.035529 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.035529 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.035513    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.136743 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.136743 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b97bf\" (UniqueName: \"kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.136943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137250 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.137140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137250 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.137220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.137376 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.137323    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.139505 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.139483    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.139750 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.139734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.144320 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.144300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b97bf\" (UniqueName: \"kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf\") pod \"ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") " pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.293725 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.293628    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:36.428605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.428576    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:08:36.431649 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:08:36.431622    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56333c31_b1da_4ec3_a0fb_d916f40ee6c9.slice/crio-bfd469d5dbca003de07d2c0f84495169c016c02dfae83e6fc2ef78f6630669cd WatchSource:0}: Error finding container bfd469d5dbca003de07d2c0f84495169c016c02dfae83e6fc2ef78f6630669cd: Status 404 returned error can't find the container with id bfd469d5dbca003de07d2c0f84495169c016c02dfae83e6fc2ef78f6630669cd
Jul 23 23:08:36.921519 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.921477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" event={"ID":"56333c31-b1da-4ec3-a0fb-d916f40ee6c9","Type":"ContainerStarted","Data":"bcacd7bfaa73273198559be3326de8d03109ac24cd320c7513db1bb96203dc9f"}
Jul 23 23:08:36.921715 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.921530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" event={"ID":"56333c31-b1da-4ec3-a0fb-d916f40ee6c9","Type":"ContainerStarted","Data":"bfd469d5dbca003de07d2c0f84495169c016c02dfae83e6fc2ef78f6630669cd"}
Jul 23 23:08:36.941425 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:36.941372    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" podStartSLOduration=1.941357473 podStartE2EDuration="1.941357473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:08:36.939475442 +0000 UTC m=+2747.303141848" watchObservedRunningTime="2026-07-23 23:08:36.941357473 +0000 UTC m=+2747.305023900"
Jul 23 23:08:39.489603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:39.489564    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:08:46.294779 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:46.294739    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:46.294779 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:46.294793    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:46.307335 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:46.307309    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:46.980879 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:46.980851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:08:58.393546 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:08:58.393504    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:09:18.052028 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:18.051945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:09:28.489555 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.489517    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"]
Jul 23 23:09:28.492659 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.492643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:28.494982 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.494952    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:09:28.494982 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.494967    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:09:28.499532 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.499508    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"]
Jul 23 23:09:28.665917 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.665868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd\" (UID: \"a837b1ea-f137-4169-b1a4-49370ead765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:28.767430 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.767342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd\" (UID: \"a837b1ea-f137-4169-b1a4-49370ead765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:28.767975 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.767957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd\" (UID: \"a837b1ea-f137-4169-b1a4-49370ead765d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:28.804404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.804362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:28.935385 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:28.935362    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"]
Jul 23 23:09:28.937417 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:09:28.937372    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda837b1ea_f137_4169_b1a4_49370ead765d.slice/crio-67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d WatchSource:0}: Error finding container 67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d: Status 404 returned error can't find the container with id 67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d
Jul 23 23:09:29.172014 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:29.171970    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd" event={"ID":"a837b1ea-f137-4169-b1a4-49370ead765d","Type":"ContainerStarted","Data":"67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d"}
Jul 23 23:09:30.178300 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:30.178262    2559 generic.go:356] "Generic (PLEG): container finished" podID="a837b1ea-f137-4169-b1a4-49370ead765d" containerID="d9b790fdb31bad45cdda4c02683b83612670720b85727b297df86c6bb1cfdc09" exitCode=0
Jul 23 23:09:30.178687 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:30.178313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd" event={"ID":"a837b1ea-f137-4169-b1a4-49370ead765d","Type":"ContainerDied","Data":"d9b790fdb31bad45cdda4c02683b83612670720b85727b297df86c6bb1cfdc09"}
Jul 23 23:09:31.311771 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:31.311749    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:31.393097 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:31.393067    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca\") pod \"a837b1ea-f137-4169-b1a4-49370ead765d\" (UID: \"a837b1ea-f137-4169-b1a4-49370ead765d\") "
Jul 23 23:09:31.393451 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:31.393424    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a837b1ea-f137-4169-b1a4-49370ead765d" (UID: "a837b1ea-f137-4169-b1a4-49370ead765d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:09:31.494661 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:31.494617    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a837b1ea-f137-4169-b1a4-49370ead765d-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:09:32.188803 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:32.188764    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"
Jul 23 23:09:32.188974 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:32.188797    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd" event={"ID":"a837b1ea-f137-4169-b1a4-49370ead765d","Type":"ContainerDied","Data":"67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d"}
Jul 23 23:09:32.188974 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:32.188856    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67c3e6c9030064f8bf2ecfbfacb29558ab18d88f016910500275a6740b230f8d"
Jul 23 23:09:43.557079 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:43.557030    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"]
Jul 23 23:09:43.561096 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:43.561070    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8300e5-nwvgd"]
Jul 23 23:09:44.110909 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:44.110868    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a837b1ea-f137-4169-b1a4-49370ead765d" path="/var/lib/kubelet/pods/a837b1ea-f137-4169-b1a4-49370ead765d/volumes"
Jul 23 23:09:45.592654 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:45.592618    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:09:53.822852 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.822817    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"]
Jul 23 23:09:53.823261 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.823189    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a837b1ea-f137-4169-b1a4-49370ead765d" containerName="revoke-keys"
Jul 23 23:09:53.823261 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.823200    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a837b1ea-f137-4169-b1a4-49370ead765d" containerName="revoke-keys"
Jul 23 23:09:53.823261 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.823258    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a837b1ea-f137-4169-b1a4-49370ead765d" containerName="revoke-keys"
Jul 23 23:09:53.827489 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.827473    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:53.829802 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.829767    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:09:53.829982 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.829961    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:09:53.834481 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.834444    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"]
Jul 23 23:09:53.916075 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:53.916037    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw\" (UID: \"946aada9-18f9-4a30-869a-cc5f137c8188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:54.017587 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:54.017548    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw\" (UID: \"946aada9-18f9-4a30-869a-cc5f137c8188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:54.018190 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:54.018170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw\" (UID: \"946aada9-18f9-4a30-869a-cc5f137c8188\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:54.147599 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:54.147512    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:54.284280 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:54.284250    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"]
Jul 23 23:09:54.289476 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:09:54.287614    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946aada9_18f9_4a30_869a_cc5f137c8188.slice/crio-3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f WatchSource:0}: Error finding container 3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f: Status 404 returned error can't find the container with id 3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f
Jul 23 23:09:54.295970 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:54.295930    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw" event={"ID":"946aada9-18f9-4a30-869a-cc5f137c8188","Type":"ContainerStarted","Data":"3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f"}
Jul 23 23:09:55.302524 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:55.302434    2559 generic.go:356] "Generic (PLEG): container finished" podID="946aada9-18f9-4a30-869a-cc5f137c8188" containerID="6a6f40efb908549a1e8446f7b77373fec8311c3a266837ee36908d06cd84ef63" exitCode=0
Jul 23 23:09:55.302870 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:55.302515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw" event={"ID":"946aada9-18f9-4a30-869a-cc5f137c8188","Type":"ContainerDied","Data":"6a6f40efb908549a1e8446f7b77373fec8311c3a266837ee36908d06cd84ef63"}
Jul 23 23:09:56.442201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:56.442172    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:56.542469 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:56.542435    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca\") pod \"946aada9-18f9-4a30-869a-cc5f137c8188\" (UID: \"946aada9-18f9-4a30-869a-cc5f137c8188\") "
Jul 23 23:09:56.542785 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:56.542764    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "946aada9-18f9-4a30-869a-cc5f137c8188" (UID: "946aada9-18f9-4a30-869a-cc5f137c8188"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:09:56.643389 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:56.643303    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aada9-18f9-4a30-869a-cc5f137c8188-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:09:57.312438 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:57.312410    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"
Jul 23 23:09:57.312438 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:57.312432    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw" event={"ID":"946aada9-18f9-4a30-869a-cc5f137c8188","Type":"ContainerDied","Data":"3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f"}
Jul 23 23:09:57.312661 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:09:57.312484    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b61b15780264c184663ecdf235984040cbb713e4f085a51c14fa1f8a721f98f"
Jul 23 23:10:09.501095 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:09.501053    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:10:13.889543 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:13.889507    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"]
Jul 23 23:10:13.891803 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:13.891779    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4b3ef9-2psjw"]
Jul 23 23:10:14.110713 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:14.110681    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946aada9-18f9-4a30-869a-cc5f137c8188" path="/var/lib/kubelet/pods/946aada9-18f9-4a30-869a-cc5f137c8188/volumes"
Jul 23 23:10:19.271171 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:19.271133    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:10:19.897104 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:19.897063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:10:34.292023 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.291965    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" podUID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" containerName="main" containerID="cri-o://bcacd7bfaa73273198559be3326de8d03109ac24cd320c7513db1bb96203dc9f" gracePeriod=45
Jul 23 23:10:34.494082 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.494053    2559 generic.go:356] "Generic (PLEG): container finished" podID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" containerID="bcacd7bfaa73273198559be3326de8d03109ac24cd320c7513db1bb96203dc9f" exitCode=0
Jul 23 23:10:34.494236 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.494109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" event={"ID":"56333c31-b1da-4ec3-a0fb-d916f40ee6c9","Type":"ContainerDied","Data":"bcacd7bfaa73273198559be3326de8d03109ac24cd320c7513db1bb96203dc9f"}
Jul 23 23:10:34.538322 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.538298    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:10:34.621523 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621424    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621523 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621608    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621637    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621724 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621697    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b97bf\" (UniqueName: \"kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621872 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621730    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache\") pod \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\" (UID: \"56333c31-b1da-4ec3-a0fb-d916f40ee6c9\") "
Jul 23 23:10:34.621957 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.621867    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:10:34.622046 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.622017    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home" (OuterVolumeSpecName: "home") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:10:34.622100 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.622046    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:10:34.622173 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.622156    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:34.622224 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.622205    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:34.622224 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.622218    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:34.623872 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.623850    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf" (OuterVolumeSpecName: "kube-api-access-b97bf") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "kube-api-access-b97bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:10:34.623960 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.623882    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:10:34.623998 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.623955    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm" (OuterVolumeSpecName: "dshm") pod "56333c31-b1da-4ec3-a0fb-d916f40ee6c9" (UID: "56333c31-b1da-4ec3-a0fb-d916f40ee6c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:10:34.723641 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.723604    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b97bf\" (UniqueName: \"kubernetes.io/projected/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-kube-api-access-b97bf\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:34.723641 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.723635    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:34.723641 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:34.723644    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56333c31-b1da-4ec3-a0fb-d916f40ee6c9-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:10:35.499619 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:35.499593    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"
Jul 23 23:10:35.499619 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:35.499600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf" event={"ID":"56333c31-b1da-4ec3-a0fb-d916f40ee6c9","Type":"ContainerDied","Data":"bfd469d5dbca003de07d2c0f84495169c016c02dfae83e6fc2ef78f6630669cd"}
Jul 23 23:10:35.500120 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:35.499651    2559 scope.go:117] "RemoveContainer" containerID="bcacd7bfaa73273198559be3326de8d03109ac24cd320c7513db1bb96203dc9f"
Jul 23 23:10:35.519568 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:35.519540    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:10:35.523248 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:35.523228    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-4b3ef9/ipp-route-4b3ef9-kserve-58cf6d9cf-7v4mf"]
Jul 23 23:10:36.109973 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:10:36.109941    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" path="/var/lib/kubelet/pods/56333c31-b1da-4ec3-a0fb-d916f40ee6c9/volumes"
Jul 23 23:11:08.485614 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.485579    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:11:08.486097 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.485815    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy" containerID="cri-o://02bb821ca8c5abb9f376daf45a8de8c2882bb3b1a97f723242dc8fa811b8e7b3" gracePeriod=30
Jul 23 23:11:08.886856 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.886768    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"]
Jul 23 23:11:08.887209 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887194    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" containerName="main"
Jul 23 23:11:08.887253 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887214    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" containerName="main"
Jul 23 23:11:08.887253 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887243    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946aada9-18f9-4a30-869a-cc5f137c8188" containerName="revoke-keys"
Jul 23 23:11:08.887253 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887249    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="946aada9-18f9-4a30-869a-cc5f137c8188" containerName="revoke-keys"
Jul 23 23:11:08.887348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887312    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="946aada9-18f9-4a30-869a-cc5f137c8188" containerName="revoke-keys"
Jul 23 23:11:08.887348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.887322    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="56333c31-b1da-4ec3-a0fb-d916f40ee6c9" containerName="main"
Jul 23 23:11:08.890293 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.890272    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:08.892612 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.892594    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:11:08.892729 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.892622    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:11:08.897192 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.897167    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"]
Jul 23 23:11:08.948030 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:08.947989    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8\" (UID: \"eaf2d0ad-9986-4889-9c79-1df4ba7ce096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:09.049111 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.049079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8\" (UID: \"eaf2d0ad-9986-4889-9c79-1df4ba7ce096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:09.049788 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.049767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8\" (UID: \"eaf2d0ad-9986-4889-9c79-1df4ba7ce096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:09.202742 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.202637    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:09.335761 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.335738    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"]
Jul 23 23:11:09.338114 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:11:09.338086    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf2d0ad_9986_4889_9c79_1df4ba7ce096.slice/crio-468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359 WatchSource:0}: Error finding container 468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359: Status 404 returned error can't find the container with id 468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359
Jul 23 23:11:09.671830 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.671790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8" event={"ID":"eaf2d0ad-9986-4889-9c79-1df4ba7ce096","Type":"ContainerStarted","Data":"1ff4b3e24c10e8e3a94cc52a9d90948ef70d18d507f664383b1777ba2b41f261"}
Jul 23 23:11:09.672241 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.671849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8" event={"ID":"eaf2d0ad-9986-4889-9c79-1df4ba7ce096","Type":"ContainerStarted","Data":"468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359"}
Jul 23 23:11:09.686100 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:09.686009    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8" podStartSLOduration=1.685990494 podStartE2EDuration="1.685990494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:11:09.685741028 +0000 UTC m=+2900.049407428" watchObservedRunningTime="2026-07-23 23:11:09.685990494 +0000 UTC m=+2900.049656877"
Jul 23 23:11:10.678146 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:10.678109    2559 generic.go:356] "Generic (PLEG): container finished" podID="eaf2d0ad-9986-4889-9c79-1df4ba7ce096" containerID="1ff4b3e24c10e8e3a94cc52a9d90948ef70d18d507f664383b1777ba2b41f261" exitCode=0
Jul 23 23:11:10.678622 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:10.678161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8" event={"ID":"eaf2d0ad-9986-4889-9c79-1df4ba7ce096","Type":"ContainerDied","Data":"1ff4b3e24c10e8e3a94cc52a9d90948ef70d18d507f664383b1777ba2b41f261"}
Jul 23 23:11:11.813447 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:11.813422    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:11.877680 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:11.877648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca\") pod \"eaf2d0ad-9986-4889-9c79-1df4ba7ce096\" (UID: \"eaf2d0ad-9986-4889-9c79-1df4ba7ce096\") "
Jul 23 23:11:11.878039 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:11.878019    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eaf2d0ad-9986-4889-9c79-1df4ba7ce096" (UID: "eaf2d0ad-9986-4889-9c79-1df4ba7ce096"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:11:11.979066 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:11.979040    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaf2d0ad-9986-4889-9c79-1df4ba7ce096-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:12.688716 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:12.688677    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8" event={"ID":"eaf2d0ad-9986-4889-9c79-1df4ba7ce096","Type":"ContainerDied","Data":"468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359"}
Jul 23 23:11:12.688716 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:12.688708    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"
Jul 23 23:11:12.688942 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:12.688739    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="468509dd433c74396857a451c55359b3055e9ad2172c2cc4ee28744d296f5359"
Jul 23 23:11:13.696454 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.696418    2559 generic.go:356] "Generic (PLEG): container finished" podID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerID="02bb821ca8c5abb9f376daf45a8de8c2882bb3b1a97f723242dc8fa811b8e7b3" exitCode=0
Jul 23 23:11:13.696818 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.696483    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerDied","Data":"02bb821ca8c5abb9f376daf45a8de8c2882bb3b1a97f723242dc8fa811b8e7b3"}
Jul 23 23:11:13.696818 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.696550    2559 scope.go:117] "RemoveContainer" containerID="3ab0311698df1fc3fc5dfd8593cf3a80498dcf5e22c93ca1e851a7bc9c75fe8b"
Jul 23 23:11:13.746596 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.746573    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:11:13.897247 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897153    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897247 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897198    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897247 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897219    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897247 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897251    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897271    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897472    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897535    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:13.897605 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897574    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx7rt\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897871 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897610    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token\") pod \"620508f3-cb97-42de-be0d-7df9dd1fea32\" (UID: \"620508f3-cb97-42de-be0d-7df9dd1fea32\") "
Jul 23 23:11:13.897871 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897616    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:13.897871 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897634    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:13.897871 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.897707    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:11:13.898134 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.898026    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620508f3-cb97-42de-be0d-7df9dd1fea32-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.898134 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.898049    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.898134 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.898065    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.898134 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.898079    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.898713 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.898684    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data" (OuterVolumeSpecName: "istio-data") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:13.899693 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.899670    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:11:13.899791 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.899717    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:13.899791 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.899776    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt" (OuterVolumeSpecName: "kube-api-access-dx7rt") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "kube-api-access-dx7rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:11:13.899919 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.899873    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token" (OuterVolumeSpecName: "istio-token") pod "620508f3-cb97-42de-be0d-7df9dd1fea32" (UID: "620508f3-cb97-42de-be0d-7df9dd1fea32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:11:13.998616 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.998574    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.998616 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.998609    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx7rt\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-kube-api-access-dx7rt\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.998616 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.998623    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.998849 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.998634    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:13.998849 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:13.998645    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620508f3-cb97-42de-be0d-7df9dd1fea32-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:14.703232 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:14.703200    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"
Jul 23 23:11:14.703232 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:14.703222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9" event={"ID":"620508f3-cb97-42de-be0d-7df9dd1fea32","Type":"ContainerDied","Data":"85b76c38fb26f2a936e7b7c08021a9935cbed7ec3f8bd64f70a51404021097cf"}
Jul 23 23:11:14.703730 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:14.703306    2559 scope.go:117] "RemoveContainer" containerID="02bb821ca8c5abb9f376daf45a8de8c2882bb3b1a97f723242dc8fa811b8e7b3"
Jul 23 23:11:14.720478 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:14.720455    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:11:14.723784 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:14.723763    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4b3ef9-openshift-default-6487bbfd99-rvrl9"]
Jul 23 23:11:16.110424 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:16.110394    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" path="/var/lib/kubelet/pods/620508f3-cb97-42de-be0d-7df9dd1fea32/volumes"
Jul 23 23:11:22.998610 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:22.998569    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:11:23.951270 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:23.951227    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"]
Jul 23 23:11:23.953879 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:23.953847    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4f67fa-dnsp8"]
Jul 23 23:11:24.109453 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:24.109421    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf2d0ad-9986-4889-9c79-1df4ba7ce096" path="/var/lib/kubelet/pods/eaf2d0ad-9986-4889-9c79-1df4ba7ce096/volumes"
Jul 23 23:11:30.938653 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.938616    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"]
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.938992    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939003    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939014    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf2d0ad-9986-4889-9c79-1df4ba7ce096" containerName="revoke-keys"
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939019    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf2d0ad-9986-4889-9c79-1df4ba7ce096" containerName="revoke-keys"
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939035    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.939038 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939040    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.939228 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939109    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf2d0ad-9986-4889-9c79-1df4ba7ce096" containerName="revoke-keys"
Jul 23 23:11:30.939228 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939118    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.939228 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.939123    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="620508f3-cb97-42de-be0d-7df9dd1fea32" containerName="istio-proxy"
Jul 23 23:11:30.942049 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.942027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:30.944432 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.944403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v6cq4\""
Jul 23 23:11:30.944581 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.944409    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:11:30.948835 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.948728    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"]
Jul 23 23:11:30.963462 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:30.963427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n\" (UID: \"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:31.064695 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.064655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n\" (UID: \"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:31.065378 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.065356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n\" (UID: \"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:31.255056 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.255010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:31.390261 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.390236    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"]
Jul 23 23:11:31.392784 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:11:31.392752    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0821bcd0_3cfc_47e4_ad8d_1c3e1c0c34bb.slice/crio-8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600 WatchSource:0}: Error finding container 8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600: Status 404 returned error can't find the container with id 8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600
Jul 23 23:11:31.791869 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.791771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n" event={"ID":"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb","Type":"ContainerStarted","Data":"df604191574f9bb0454d068ebc6a2ec52b110bb894a94a851c98e5e66910d45b"}
Jul 23 23:11:31.791869 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.791821    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n" event={"ID":"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb","Type":"ContainerStarted","Data":"8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600"}
Jul 23 23:11:31.807313 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:31.807262    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n" podStartSLOduration=1.807246664 podStartE2EDuration="1.807246664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:11:31.805192423 +0000 UTC m=+2922.168858833" watchObservedRunningTime="2026-07-23 23:11:31.807246664 +0000 UTC m=+2922.170913048"
Jul 23 23:11:32.798543 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:32.798511    2559 generic.go:356] "Generic (PLEG): container finished" podID="0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" containerID="df604191574f9bb0454d068ebc6a2ec52b110bb894a94a851c98e5e66910d45b" exitCode=0
Jul 23 23:11:32.798966 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:32.798587    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n" event={"ID":"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb","Type":"ContainerDied","Data":"df604191574f9bb0454d068ebc6a2ec52b110bb894a94a851c98e5e66910d45b"}
Jul 23 23:11:33.959214 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:33.959188    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:33.993987 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:33.993951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca\") pod \"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb\" (UID: \"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb\") "
Jul 23 23:11:33.994308 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:33.994286    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" (UID: "0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:11:34.095812 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:34.095725    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:34.810872 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:34.810830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n" event={"ID":"0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb","Type":"ContainerDied","Data":"8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600"}
Jul 23 23:11:34.811061 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:34.810905    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f3dc1299bd88a82521bf2812cf976de778029034f79c8ef1d718a06b71d3600"
Jul 23 23:11:34.811061 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:34.810857    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"
Jul 23 23:11:44.387864 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:44.387833    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:11:46.003397 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:46.003336    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"]
Jul 23 23:11:46.005037 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:46.005013    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fd1be5-4b55n"]
Jul 23 23:11:46.109769 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:46.109726    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" path="/var/lib/kubelet/pods/0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb/volumes"
Jul 23 23:11:52.589352 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:52.589318    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:11:52.589761 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:52.589568    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy" containerID="cri-o://97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092" gracePeriod=30
Jul 23 23:11:57.859685 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.859622    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:11:57.921259 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.921222    2559 generic.go:356] "Generic (PLEG): container finished" podID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerID="97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092" exitCode=0
Jul 23 23:11:57.921460 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.921285    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerDied","Data":"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"}
Jul 23 23:11:57.921460 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.921289    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"
Jul 23 23:11:57.921460 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.921328    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx" event={"ID":"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c","Type":"ContainerDied","Data":"e9da2fe803974ce6c5c4c29cab301422d2cf48f6888298922e24cd6f984f29b8"}
Jul 23 23:11:57.921460 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.921350    2559 scope.go:117] "RemoveContainer" containerID="97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"
Jul 23 23:11:57.929864 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.929846    2559 scope.go:117] "RemoveContainer" containerID="472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"
Jul 23 23:11:57.937698 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.937680    2559 scope.go:117] "RemoveContainer" containerID="97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"
Jul 23 23:11:57.937988 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:11:57.937964    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092\": container with ID starting with 97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092 not found: ID does not exist" containerID="97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"
Jul 23 23:11:57.938066 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.937997    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092"} err="failed to get container status \"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092\": rpc error: code = NotFound desc = could not find container \"97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092\": container with ID starting with 97dd193608d551119a5b61fb03d8f3906e32e7bc406e11137533d1dbc3c7a092 not found: ID does not exist"
Jul 23 23:11:57.938066 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.938015    2559 scope.go:117] "RemoveContainer" containerID="472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"
Jul 23 23:11:57.938305 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:11:57.938285    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506\": container with ID starting with 472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506 not found: ID does not exist" containerID="472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"
Jul 23 23:11:57.938368 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:57.938313    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506"} err="failed to get container status \"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506\": rpc error: code = NotFound desc = could not find container \"472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506\": container with ID starting with 472e1dc6920170691604b1458bfaed9fb6bf9ddab8fd4a89355bc0d8a5488506 not found: ID does not exist"
Jul 23 23:11:58.022571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022537    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022571 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022577    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022767 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022596    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022767 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022767 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022650    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022767 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.022767 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022741    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.023063 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022801    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmps9\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.023063 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022872    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket\") pod \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\" (UID: \"ca6f79fb-b04d-48f5-bf67-7fd8b81e506c\") "
Jul 23 23:11:58.023063 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.022884    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:58.023215 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023120    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:58.023267 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023228    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:11:58.023267 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023253    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.023357 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023274    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.023357 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023315    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:58.023924 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.023795    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:58.025094 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.025065    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:11:58.025290 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.025264    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9" (OuterVolumeSpecName: "kube-api-access-qmps9") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "kube-api-access-qmps9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:11:58.025518 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.025501    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:11:58.025579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.025509    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" (UID: "ca6f79fb-b04d-48f5-bf67-7fd8b81e506c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:11:58.124274 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124250    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124274 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124275    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124284    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124292    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124301    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124307    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.124404 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.124315    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmps9\" (UniqueName: \"kubernetes.io/projected/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c-kube-api-access-qmps9\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:11:58.237370 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.237328    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:11:58.243221 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:11:58.243196    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-fd1be5-openshift-default-76b46bbb9f-q75kx"]
Jul 23 23:12:00.109664 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:00.109628    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" path="/var/lib/kubelet/pods/ca6f79fb-b04d-48f5-bf67-7fd8b81e506c/volumes"
Jul 23 23:12:06.297490 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:06.297447    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmhsn"]
Jul 23 23:12:14.779561 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:14.779487    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:12:14.780031 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:14.779692    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy" containerID="cri-o://4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540" gracePeriod=30
Jul 23 23:12:20.034107 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.034081    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:12:20.036042 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.036019    2559 generic.go:356] "Generic (PLEG): container finished" podID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerID="4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540" exitCode=0
Jul 23 23:12:20.036141 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.036061    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"
Jul 23 23:12:20.036141 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.036076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerDied","Data":"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"}
Jul 23 23:12:20.036141 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.036117    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr" event={"ID":"62bf818c-09a3-42d1-94c8-e1eae259606a","Type":"ContainerDied","Data":"a1b686240ea1590894c8330bed06a391374f92bfe0fb6b8ad4af66eddac2241a"}
Jul 23 23:12:20.036141 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.036139    2559 scope.go:117] "RemoveContainer" containerID="4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"
Jul 23 23:12:20.044484 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.044459    2559 scope.go:117] "RemoveContainer" containerID="fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"
Jul 23 23:12:20.052295 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.052276    2559 scope.go:117] "RemoveContainer" containerID="4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"
Jul 23 23:12:20.052558 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:12:20.052533    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540\": container with ID starting with 4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540 not found: ID does not exist" containerID="4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"
Jul 23 23:12:20.052683 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.052569    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540"} err="failed to get container status \"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540\": rpc error: code = NotFound desc = could not find container \"4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540\": container with ID starting with 4452226a4bbbfff4b2c0a8ed53cd040dbd70934f0aa1a3dba1a1e56fa37c6540 not found: ID does not exist"
Jul 23 23:12:20.052765 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.052689    2559 scope.go:117] "RemoveContainer" containerID="fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"
Jul 23 23:12:20.053663 ip-10-0-133-188 kubenswrapper[2559]: E0723 23:12:20.053585    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab\": container with ID starting with fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab not found: ID does not exist" containerID="fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"
Jul 23 23:12:20.053663 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.053623    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab"} err="failed to get container status \"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab\": rpc error: code = NotFound desc = could not find container \"fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab\": container with ID starting with fa0edbb94e2ab0f47a5a94ae68e6933ea3bfa48b1341b73b6faf932a55ff18ab not found: ID does not exist"
Jul 23 23:12:20.121165 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121126    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121362 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121362 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121218    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121362 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121248    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121362 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121368    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121400    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9ds4\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121446    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121457    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:12:20.121579 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121474    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs\") pod \"62bf818c-09a3-42d1-94c8-e1eae259606a\" (UID: \"62bf818c-09a3-42d1-94c8-e1eae259606a\") "
Jul 23 23:12:20.121827 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121588    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:12:20.121827 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121693    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:12:20.121827 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.121808    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:12:20.122045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.122025    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.122276 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.122052    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.122276 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.122066    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62bf818c-09a3-42d1-94c8-e1eae259606a-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.122276 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.122078    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.122604 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.122569    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data" (OuterVolumeSpecName: "istio-data") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:12:20.123734 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.123702    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token" (OuterVolumeSpecName: "istio-token") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:12:20.123853 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.123754    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:12:20.123853 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.123776    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4" (OuterVolumeSpecName: "kube-api-access-p9ds4") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "kube-api-access-p9ds4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:12:20.123853 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.123791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62bf818c-09a3-42d1-94c8-e1eae259606a" (UID: "62bf818c-09a3-42d1-94c8-e1eae259606a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:12:20.223479 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.223436    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.223479 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.223468    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.223479 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.223482    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.223732 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.223495    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9ds4\" (UniqueName: \"kubernetes.io/projected/62bf818c-09a3-42d1-94c8-e1eae259606a-kube-api-access-p9ds4\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.223732 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.223506    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62bf818c-09a3-42d1-94c8-e1eae259606a-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Jul 23 23:12:20.356363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.356332    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:12:20.359917 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:20.359874    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-159535-openshift-default-66f49966ff-lxrfr"]
Jul 23 23:12:22.025446 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:22.025415    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fc7cc8c86-zqkh2_5f953a44-525d-4a52-b20c-5207460cbf28/copy-manifests/0.log"
Jul 23 23:12:22.032412 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:22.032388    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fc7cc8c86-zqkh2_5f953a44-525d-4a52-b20c-5207460cbf28/manager/0.log"
Jul 23 23:12:22.110058 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:22.110022    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" path="/var/lib/kubelet/pods/62bf818c-09a3-42d1-94c8-e1eae259606a/volumes"
Jul 23 23:12:22.625910 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:22.625866    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747460-nb29n_b0ea1a06-d144-4229-b389-92c33d7ff0e9/cleanup/0.log"
Jul 23 23:12:24.830309 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:24.830276    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmhsn_340a1ed3-110e-4a6d-9b3f-198076a572f8/limitador/0.log"
Jul 23 23:12:25.330279 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:25.330242    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wcqz4_96f4d15f-289c-4d09-bbb6-5beebd242e91/istio-proxy/0.log"
Jul 23 23:12:25.573788 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:25.573761    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-699c848bdc-6d9cc_daade4cc-dddb-4fb9-a2b1-cab1e120252a/kube-auth-proxy/0.log"
Jul 23 23:12:25.822047 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:25.822016    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/1.log"
Jul 23 23:12:27.169026 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:27.168994    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-524rx_782484c3-e25b-472d-836a-55f797fc5805/main/0.log"
Jul 23 23:12:32.693553 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693520    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbzbm/must-gather-f4m94"]
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693846    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" containerName="revoke-keys"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693857    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" containerName="revoke-keys"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693867    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693873    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693881    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693887    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693912    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693918    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693966    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693973    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693980    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0821bcd0-3cfc-47e4-ad8d-1c3e1c0c34bb" containerName="revoke-keys"
Jul 23 23:12:32.694009 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.693986    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694378 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.694032    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694378 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.694037    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bf818c-09a3-42d1-94c8-e1eae259606a" containerName="istio-proxy"
Jul 23 23:12:32.694378 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.694097    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6f79fb-b04d-48f5-bf67-7fd8b81e506c" containerName="istio-proxy"
Jul 23 23:12:32.697089 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.697068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.699477 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.699448    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbzbm\"/\"openshift-service-ca.crt\""
Jul 23 23:12:32.699477 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.699466    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbzbm\"/\"kube-root-ca.crt\""
Jul 23 23:12:32.700255 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.700240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vbzbm\"/\"default-dockercfg-5tpj8\""
Jul 23 23:12:32.706045 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.706023    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbzbm/must-gather-f4m94"]
Jul 23 23:12:32.839957 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.839917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldlwt\" (UniqueName: \"kubernetes.io/projected/1da450e3-a56b-4c6d-a16f-00b37946adfd-kube-api-access-ldlwt\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.840143 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.839987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1da450e3-a56b-4c6d-a16f-00b37946adfd-must-gather-output\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.940587 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.940550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1da450e3-a56b-4c6d-a16f-00b37946adfd-must-gather-output\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.940758 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.940627    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldlwt\" (UniqueName: \"kubernetes.io/projected/1da450e3-a56b-4c6d-a16f-00b37946adfd-kube-api-access-ldlwt\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.940924 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.940877    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1da450e3-a56b-4c6d-a16f-00b37946adfd-must-gather-output\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:32.948208 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:32.948146    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldlwt\" (UniqueName: \"kubernetes.io/projected/1da450e3-a56b-4c6d-a16f-00b37946adfd-kube-api-access-ldlwt\") pod \"must-gather-f4m94\" (UID: \"1da450e3-a56b-4c6d-a16f-00b37946adfd\") " pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:33.007201 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:33.007164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbzbm/must-gather-f4m94"
Jul 23 23:12:33.136239 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:33.136209    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbzbm/must-gather-f4m94"]
Jul 23 23:12:33.139577 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:12:33.139547    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1da450e3_a56b_4c6d_a16f_00b37946adfd.slice/crio-abaa9b3c72a493d680e6893e31b23c57c2e7c6c8c26bc39a2b851e2a30db1ef4 WatchSource:0}: Error finding container abaa9b3c72a493d680e6893e31b23c57c2e7c6c8c26bc39a2b851e2a30db1ef4: Status 404 returned error can't find the container with id abaa9b3c72a493d680e6893e31b23c57c2e7c6c8c26bc39a2b851e2a30db1ef4
Jul 23 23:12:33.141459 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:33.141443    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:12:34.111195 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:34.111152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbzbm/must-gather-f4m94" event={"ID":"1da450e3-a56b-4c6d-a16f-00b37946adfd","Type":"ContainerStarted","Data":"abaa9b3c72a493d680e6893e31b23c57c2e7c6c8c26bc39a2b851e2a30db1ef4"}
Jul 23 23:12:35.116354 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:35.116307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbzbm/must-gather-f4m94" event={"ID":"1da450e3-a56b-4c6d-a16f-00b37946adfd","Type":"ContainerStarted","Data":"d1fef8563ad33d55e9fd70bdfe7e046285531941b52925c496590c62b8497566"}
Jul 23 23:12:35.116843 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:35.116367    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbzbm/must-gather-f4m94" event={"ID":"1da450e3-a56b-4c6d-a16f-00b37946adfd","Type":"ContainerStarted","Data":"91608edbb65ac7a77a03ca62e9ca7e351e49ef5a7c3b01ecc600a60d7a5b3b49"}
Jul 23 23:12:35.132228 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:35.132176    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbzbm/must-gather-f4m94" podStartSLOduration=2.065515673 podStartE2EDuration="3.132159319s" totalImagesPullingTime="1.066643646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:12:35.131116016 +0000 UTC m=+2985.494782400" watchObservedRunningTime="2026-07-23 23:12:35.132159319 +0000 UTC m=+2985.495825702"
Jul 23 23:12:35.874105 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:35.874075    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x5h2b_8dbe3df2-31d4-4acf-abb7-04f4fa745718/global-pull-secret-syncer/0.log"
Jul 23 23:12:35.963827 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:35.963794    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-74qqm_9ee10fcc-cfb7-4b3c-8c72-19ecf031ac39/konnectivity-agent/0.log"
Jul 23 23:12:36.040603 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:36.040571    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-188.ec2.internal_36e80edc15aa7c49c99a25a6d073ca65/haproxy/0.log"
Jul 23 23:12:41.468178 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:41.468146    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmhsn_340a1ed3-110e-4a6d-9b3f-198076a572f8/limitador/0.log"
Jul 23 23:12:43.548036 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.548008    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5549556469-s828j_45567508-5e3d-4f78-9d04-00d6801a6bc4/metrics-server/0.log"
Jul 23 23:12:43.577301 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.577207    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-kqkm5_1e7537ff-55d1-4e98-840f-728ec741c417/monitoring-plugin/0.log"
Jul 23 23:12:43.678388 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.678352    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc9kp_b42c2bf4-7109-4684-b164-bc25e8c65155/node-exporter/0.log"
Jul 23 23:12:43.701237 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.701196    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc9kp_b42c2bf4-7109-4684-b164-bc25e8c65155/kube-rbac-proxy/0.log"
Jul 23 23:12:43.723554 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.723520    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc9kp_b42c2bf4-7109-4684-b164-bc25e8c65155/init-textfile/0.log"
Jul 23 23:12:43.919326 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.919232    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/prometheus/0.log"
Jul 23 23:12:43.941628 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.941596    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/config-reloader/0.log"
Jul 23 23:12:43.962113 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.962082    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/thanos-sidecar/0.log"
Jul 23 23:12:43.983184 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:43.983146    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/kube-rbac-proxy-web/0.log"
Jul 23 23:12:44.003751 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.003717    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/kube-rbac-proxy/0.log"
Jul 23 23:12:44.025772 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.025745    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/kube-rbac-proxy-thanos/0.log"
Jul 23 23:12:44.046436 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.046245    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a561679a-85b0-4e7b-954a-0b134bb1cfc4/init-config-reloader/0.log"
Jul 23 23:12:44.075348 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.075296    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-d5ztq_908418ef-7e8b-4b18-a63e-b23d6ab69739/prometheus-operator/0.log"
Jul 23 23:12:44.092610 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.092586    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-d5ztq_908418ef-7e8b-4b18-a63e-b23d6ab69739/kube-rbac-proxy/0.log"
Jul 23 23:12:44.238914 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.238864    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/thanos-query/0.log"
Jul 23 23:12:44.260112 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.260083    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/kube-rbac-proxy-web/0.log"
Jul 23 23:12:44.281100 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.281073    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/kube-rbac-proxy/0.log"
Jul 23 23:12:44.302773 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.302741    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/prom-label-proxy/0.log"
Jul 23 23:12:44.321958 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.321927    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/kube-rbac-proxy-rules/0.log"
Jul 23 23:12:44.342186 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:44.342153    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8589f78d67-q2ncd_bc2563dd-35e1-4328-b8db-827c3c353f6b/kube-rbac-proxy-metrics/0.log"
Jul 23 23:12:46.173563 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.173526    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"]
Jul 23 23:12:46.180831 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.180798    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.189678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.188189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-lib-modules\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.189678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.188311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-podres\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.189678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.188350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-proc\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.189678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.188429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmtr\" (UniqueName: \"kubernetes.io/projected/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-kube-api-access-9vmtr\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.189678 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.188471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-sys\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.194563 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.193681    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"]
Jul 23 23:12:46.289569 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.289523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-sys\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.289840 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.289819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-lib-modules\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.290044 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.290028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-podres\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.290171 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.290158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-proc\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.290334 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.290317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmtr\" (UniqueName: \"kubernetes.io/projected/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-kube-api-access-9vmtr\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.290875 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.290850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-sys\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.291164 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.291147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-lib-modules\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.291347 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.291327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-podres\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.291484 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.291471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-proc\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.299929 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.299708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmtr\" (UniqueName: \"kubernetes.io/projected/7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486-kube-api-access-9vmtr\") pod \"perf-node-gather-daemonset-sjhhw\" (UID: \"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486\") " pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.502734 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.502696    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:46.692908 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.692684    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"]
Jul 23 23:12:46.697334 ip-10-0-133-188 kubenswrapper[2559]: W0723 23:12:46.697196    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7ba5b6f3_b44d_4b1d_b83f_e3e9b607d486.slice/crio-120e9a7d6a9025d6380dec4a04a738080df4d4f2593d75fb2e017b7c53d4b29e WatchSource:0}: Error finding container 120e9a7d6a9025d6380dec4a04a738080df4d4f2593d75fb2e017b7c53d4b29e: Status 404 returned error can't find the container with id 120e9a7d6a9025d6380dec4a04a738080df4d4f2593d75fb2e017b7c53d4b29e
Jul 23 23:12:46.802288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.802259    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68c458576d-wjvdc_452577b2-918a-4ed1-afef-09b8174cca62/console/0.log"
Jul 23 23:12:46.844145 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:46.844014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-7p69h_c6a4b1ef-5229-4d78-a855-1c3fe7753f73/download-server/0.log"
Jul 23 23:12:47.207288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:47.207162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw" event={"ID":"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486","Type":"ContainerStarted","Data":"1e9a98a4ddd14cc0e0b3a0629f3794252d19944efa276a9d6badccc5b51bd6e6"}
Jul 23 23:12:47.207288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:47.207228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw" event={"ID":"7ba5b6f3-b44d-4b1d-b83f-e3e9b607d486","Type":"ContainerStarted","Data":"120e9a7d6a9025d6380dec4a04a738080df4d4f2593d75fb2e017b7c53d4b29e"}
Jul 23 23:12:47.207288 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:47.207257    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:47.225032 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:47.224966    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw" podStartSLOduration=1.224943006 podStartE2EDuration="1.224943006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:12:47.221574176 +0000 UTC m=+2997.585240583" watchObservedRunningTime="2026-07-23 23:12:47.224943006 +0000 UTC m=+2997.588609390"
Jul 23 23:12:48.690819 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:48.690784    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9mlv_12618903-1464-4a3c-afd1-0d0f136489b4/dns/0.log"
Jul 23 23:12:48.709674 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:48.709650    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9mlv_12618903-1464-4a3c-afd1-0d0f136489b4/kube-rbac-proxy/0.log"
Jul 23 23:12:48.774269 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:48.774233    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4t9t_d55a6530-2c75-405e-b5d8-9ccc3b5660b7/dns-node-resolver/0.log"
Jul 23 23:12:49.355275 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:49.355219    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdgfz_36bec428-512b-40ae-b77a-eb0dbbd99810/node-ca/0.log"
Jul 23 23:12:50.297745 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:50.297695    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wcqz4_96f4d15f-289c-4d09-bbb6-5beebd242e91/istio-proxy/0.log"
Jul 23 23:12:50.469611 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:50.469551    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-699c848bdc-6d9cc_daade4cc-dddb-4fb9-a2b1-cab1e120252a/kube-auth-proxy/0.log"
Jul 23 23:12:50.526259 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:50.526230    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/0.log"
Jul 23 23:12:50.587196 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:50.587103    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/1.log"
Jul 23 23:12:51.471090 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:51.471063    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2wl7f_148a0ac9-e46b-4ef7-b285-278e4781d905/serve-healthcheck-canary/0.log"
Jul 23 23:12:51.993948 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:51.993916    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289lh_c5a62325-5e79-423c-8be1-7f2064e1ecc1/kube-rbac-proxy/0.log"
Jul 23 23:12:52.014967 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:52.014934    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289lh_c5a62325-5e79-423c-8be1-7f2064e1ecc1/exporter/0.log"
Jul 23 23:12:52.037052 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:52.037018    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289lh_c5a62325-5e79-423c-8be1-7f2064e1ecc1/extractor/0.log"
Jul 23 23:12:53.225342 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:53.225155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vbzbm/perf-node-gather-daemonset-sjhhw"
Jul 23 23:12:54.816599 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:54.816554    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fc7cc8c86-zqkh2_5f953a44-525d-4a52-b20c-5207460cbf28/manager/0.log"
Jul 23 23:12:54.835363 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:54.835336    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fc7cc8c86-zqkh2_5f953a44-525d-4a52-b20c-5207460cbf28/copy-manifests/0.log"
Jul 23 23:12:56.260145 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:56.260113    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58df94799c-x6s5t_38bedc5a-d031-43b3-bbd5-92040a855d0f/manager/0.log"
Jul 23 23:12:56.286342 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:56.286300    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-slzcj_0e4e3cc8-69b5-444b-84e7-b11a3953cc75/openshift-lws-operator/0.log"
Jul 23 23:12:56.904208 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:56.904183    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wcqz4_96f4d15f-289c-4d09-bbb6-5beebd242e91/istio-proxy/0.log"
Jul 23 23:12:57.073507 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:57.073478    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-699c848bdc-6d9cc_daade4cc-dddb-4fb9-a2b1-cab1e120252a/kube-auth-proxy/0.log"
Jul 23 23:12:57.108318 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:57.108276    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/0.log"
Jul 23 23:12:57.167074 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:57.166992    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/1.log"
Jul 23 23:12:58.685353 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:58.685323    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wcqz4_96f4d15f-289c-4d09-bbb6-5beebd242e91/istio-proxy/0.log"
Jul 23 23:12:58.835258 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:58.835229    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-699c848bdc-6d9cc_daade4cc-dddb-4fb9-a2b1-cab1e120252a/kube-auth-proxy/0.log"
Jul 23 23:12:58.883496 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:58.883357    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/0.log"
Jul 23 23:12:58.950187 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:12:58.950091    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2265k_98b5d6ad-9f47-479b-9d42-10782203afce/istio-proxy/1.log"
Jul 23 23:13:02.774864 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:02.774831    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5qjfk_aaedd58c-f508-49aa-96a0-cc1308fcd8c5/kube-multus/0.log"
Jul 23 23:13:02.955121 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:02.955088    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/kube-multus-additional-cni-plugins/0.log"
Jul 23 23:13:02.974508 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:02.974475    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/egress-router-binary-copy/0.log"
Jul 23 23:13:02.994130 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:02.994099    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/cni-plugins/0.log"
Jul 23 23:13:03.015179 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.015146    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/bond-cni-plugin/0.log"
Jul 23 23:13:03.036491 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.036466    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/routeoverride-cni/0.log"
Jul 23 23:13:03.057150 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.057115    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/whereabouts-cni-bincopy/0.log"
Jul 23 23:13:03.076227 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.076202    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8pl4b_e07a8b0e-1d52-4822-8d02-4a7f82b83486/whereabouts-cni/0.log"
Jul 23 23:13:03.368793 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.368710    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pzvld_ef8a2fa6-bcad-403b-a97a-bc833bd9586c/network-metrics-daemon/0.log"
Jul 23 23:13:03.383684 ip-10-0-133-188 kubenswrapper[2559]: I0723 23:13:03.383654    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pzvld_ef8a2fa6-bcad-403b-a97a-bc833bd9586c/kube-rbac-proxy/0.log"