Jul 30 00:41:25.444692 ip-10-0-134-28 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 30 00:41:25.444709 ip-10-0-134-28 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 30 00:41:25.444719 ip-10-0-134-28 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 30 00:41:25.445045 ip-10-0-134-28 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 30 00:41:35.456957 ip-10-0-134-28 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 30 00:41:35.456976 ip-10-0-134-28 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1068cc70ed8b48c0b2ae6cd2a1e1b401 --
Jul 30 00:43:31.315917 ip-10-0-134-28 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 00:43:31.731902 ip-10-0-134-28 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:31.731902 ip-10-0-134-28 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 00:43:31.731902 ip-10-0-134-28 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:31.731902 ip-10-0-134-28 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 00:43:31.731902 ip-10-0-134-28 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:31.733948 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.733850    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 00:43:31.739075 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739057    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.739075 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739075    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739079    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739082    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739085    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739088    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739091    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739093    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739096    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739099    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739101    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739104    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739107    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739109    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739113    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739116    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739119    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739122    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739125    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739134    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739137    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.739146 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739140    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739142    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739145    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739148    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739151    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739154    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739158    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739160    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739163    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739166    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739169    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739171    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739174    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739176    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739179    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739181    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739184    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739186    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739189    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739191    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.739719 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739196    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739200    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739204    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739206    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739209    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739212    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739217    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739219    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739222    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739225    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739228    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739230    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739233    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739238    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739241    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739244    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739247    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739250    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739253    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.740227 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739256    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739258    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739261    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739264    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739267    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739270    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739272    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739275    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739277    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739283    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739287    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739290    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739292    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739295    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739298    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739300    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739303    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739305    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739308    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739312    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.740753 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739315    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739318    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739321    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739323    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739326    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739328    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739331    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739334    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739336    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739339    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739342    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739344    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739347    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739349    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739352    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739355    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739357    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739360    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.739362    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740087    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.741262 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740094    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740097    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740100    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740103    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740106    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740109    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740112    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740116    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740120    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740122    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740125    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740128    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740131    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740134    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740137    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740140    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740142    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740145    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740148    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740151    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.741769 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740153    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740169    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740172    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740175    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740178    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740181    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740184    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740187    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740189    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740192    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740196    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740200    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740203    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740206    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740209    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740214    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740217    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740220    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740222    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.742269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740225    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740228    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740231    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740234    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740237    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740240    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740243    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740245    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740248    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740251    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740254    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740256    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740260    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740262    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740265    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740267    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740270    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740273    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740275    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740278    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.742756 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740280    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740283    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740285    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740288    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740290    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740293    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740295    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740298    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740301    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740304    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740306    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740309    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740311    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740313    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740316    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740320    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740322    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740325    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740327    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740330    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.743265 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740332    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740335    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740337    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740340    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740343    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740346    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740349    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740351    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740354    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740357    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740359    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740362    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740366    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740369    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740372    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740375    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740378    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740381    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740384    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.743787 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740789    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740795    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740799    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740802    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740804    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740807    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740810    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740812    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740815    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740818    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740821    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740823    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740826    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740829    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740831    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740834    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740837    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740839    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740842    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.744269 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740845    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740849    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740853    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740855    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740858    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740861    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740864    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740866    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740869    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740871    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740874    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740876    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740879    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740881    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740883    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740886    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740890    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740893    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740897    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.744755 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740899    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740902    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740905    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740908    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740911    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740913    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740916    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740918    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740921    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740924    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740926    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740931    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740934    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740938    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740941    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740944    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740946    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740949    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740951    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740954    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.745219 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740956    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740959    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740961    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740963    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740966    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740968    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740971    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740973    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740976    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740978    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740981    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740984    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740987    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740989    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740992    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740994    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.740997    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741000    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741003    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741005    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.745774 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741008    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741010    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741016    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741018    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741020    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741023    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741025    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741028    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741030    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741033    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741035    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741038    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741040    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741043    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741047    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741049    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741052    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741054    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741057    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741060    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.746263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741063    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741134    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741147    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741156    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741163    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741170    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741175    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741186    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741198    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741201    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741204    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741208    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741212    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741215    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741220    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741223    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741226    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741229    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741232    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741238    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741241    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741244    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741247    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741250    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 00:43:31.746783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741254    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741257    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741261    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741264    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741268    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741271    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741274    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741277    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741280    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741285    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741288    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741291    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741294    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741298    2584 flags.go:64] FLAG: --enable-server="true"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741301    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741305    2584 flags.go:64] FLAG: --event-burst="100"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741309    2584 flags.go:64] FLAG: --event-qps="50"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741311    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741314    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741317    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741321    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741324    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741328    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741331    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741334    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 00:43:31.747382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741337    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741340    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741342    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741345    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741348    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741351    2584 flags.go:64] FLAG: --feature-gates=""
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741355    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741358    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741361    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741364    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741367    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741370    2584 flags.go:64] FLAG: --help="false"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741373    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-134-28.ec2.internal"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741376    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741379    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741382    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741385    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741389    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741392    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741394    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741398    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741401    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741404    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741407    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 00:43:31.748025 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741410    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741413    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741416    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741419    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741422    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741426    2584 flags.go:64] FLAG: --lock-file=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741429    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741432    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741436    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741441    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741444    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741447    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741450    2584 flags.go:64] FLAG: --logging-format="text"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741453    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741456    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741459    2584 flags.go:64] FLAG: --manifest-url=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741461    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741466    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741469    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741473    2584 flags.go:64] FLAG: --max-pods="110"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741476    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741479    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741482    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741485    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741488    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 00:43:31.748969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741491    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741494    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741501    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741504    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741509    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741513    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741516    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741522    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741525    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741528    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741531    2584 flags.go:64] FLAG: --port="10250"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741534    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741538    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b47d39a055f953e"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741542    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741544    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741547    2584 flags.go:64] FLAG: --register-node="true"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741550    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741554    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741557    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741559    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741562    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741569    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741572    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741575    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741578    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 00:43:31.750096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741580    2584 flags.go:64] FLAG: --runonce="false"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741583    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741586    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741589    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741592    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741595    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741598    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741601    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741604    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741607    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741610    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741613    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741616    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741619    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741622    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741625    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741642    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741645    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741648    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741656    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741659    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741662    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741665    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741668    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741671    2584 flags.go:64] FLAG: --v="2"
Jul 30 00:43:31.751082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741675    2584 flags.go:64] FLAG: --version="false"
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741679    2584 flags.go:64] FLAG: --vmodule=""
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741682    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.741686    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741805    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741810    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741813    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741816    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741819    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741821    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741825    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741827    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741830    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741833    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741835    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741838    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741840    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741843    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741847    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741850    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.751968 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741853    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741856    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741858    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741861    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741864    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741867    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741870    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741873    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741875    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741878    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741881    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741883    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741886    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741888    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741891    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741893    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741896    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741899    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741901    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741905    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.752543 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741907    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741911    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741915    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741918    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741920    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741923    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741927    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741931    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741934    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741937    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741941    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741943    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741946    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741949    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741952    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741954    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741957    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741960    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741962    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.753153 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741965    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741968    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741970    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741973    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741976    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741979    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741981    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741984    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741986    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741989    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741992    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741994    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.741997    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742000    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742002    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742005    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742007    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742010    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742013    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742016    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.753659 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742018    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742021    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742023    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742027    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742030    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742033    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742036    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742039    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742042    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742045    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742048    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742050    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742053    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742055    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742058    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742061    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742063    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742066    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742068    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.754163 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742072    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742075    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742078    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742081    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.742084    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.742905    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.752037    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.752059    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.752944    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.752971    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753273    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753283    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753286    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753290    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753292    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.754763 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753295    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753298    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753301    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753303    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753306    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753309    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753312    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753314    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753317    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753319    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753322    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753324    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753327    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753330    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753332    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753335    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753337    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753340    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753342    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753345    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.755149 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753347    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753350    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753353    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753356    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753358    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753360    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753368    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753371    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753374    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753377    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753379    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753382    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753385    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753387    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753390    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753392    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753395    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753397    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753402    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753405    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.755648 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753410    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753412    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753417    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753419    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753424    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753426    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753429    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753432    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753434    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753437    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753439    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753442    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753444    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753447    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753449    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753452    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753455    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753458    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753461    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753464    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.756140 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753467    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753469    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753472    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753475    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753477    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753480    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753484    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753488    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753491    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753494    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753496    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753499    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753501    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753504    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753507    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753510    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753512    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753515    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753517    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.756685 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753519    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753522    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753525    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753527    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753530    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753532    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753535    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753537    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753540    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753543    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753545    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753550    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753555    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753559    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.757159 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753562    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.753568    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753729    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753734    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753737    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753739    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753742    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753745    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753748    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753750    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753753    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753756    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753759    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753761    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753764    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:31.757508 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753767    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753769    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753772    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753775    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753777    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753780    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753784    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753787    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753791    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753794    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753797    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753799    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753802    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753804    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753808    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753811    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753813    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753816    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753818    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:31.757905 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753821    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753823    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753826    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753828    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753831    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753834    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753836    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753838    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753841    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753843    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753846    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753849    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753851    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753855    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753858    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753861    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753863    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753866    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753870    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:31.758397 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753873    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753876    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753879    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753882    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753884    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753887    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753890    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753892    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753895    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753897    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753900    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753902    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753905    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753907    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753910    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753912    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753915    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753917    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753920    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753922    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753925    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:31.758937 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753927    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753930    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753932    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753935    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753937    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753941    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753945    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753948    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753951    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753953    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753956    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753958    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753961    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753964    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753966    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753969    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753971    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753973    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753976    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753978    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:31.759453 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753981    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753984    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753986    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753989    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753991    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753993    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:31.753996    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.754001    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.754120    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.757154    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.758617    2584 server.go:1019] "Starting client certificate rotation"
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.758765    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 00:43:31.759947 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.758794    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 00:43:31.761272 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.761259    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 00:43:31.764384 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.762870    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 00:43:31.773860 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.773839    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 30 00:43:31.776082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.776066    2584 log.go:25] "Validated CRI v1 image API"
Jul 30 00:43:31.776991 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.776976    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 00:43:31.781084 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.781064    2584 fs.go:135] Filesystem UUIDs: map[4caee6a7-6664-4a26-b8aa-f4a3d660d40f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f065b7c7-45b9-4214-a594-c18edf40eb0b:/dev/nvme0n1p4]
Jul 30 00:43:31.781147 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.781084    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 00:43:31.786555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.786439    2584 manager.go:216] Machine: {Timestamp:2026-07-30 00:43:31.785154955 +0000 UTC m=+0.344269865 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100344 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239d71891e6c1ca2a6fd3dfff4cf62 SystemUUID:ec239d71-891e-6c1c-a2a6-fd3dfff4cf62 BootID:1068cc70-ed8b-48c0-b2ae-6cd2a1e1b401 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:70:55:09:e0:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:70:55:09:e0:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:84:4d:7f:d6:52 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 00:43:31.786555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.786549    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 00:43:31.786702 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.786674    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 00:43:31.788404 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.788387    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 00:43:31.789724 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.789698    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 00:43:31.789892 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.789727    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 00:43:31.790565 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.790554    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 00:43:31.790601 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.790567    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 00:43:31.790601 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.790575    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 00:43:31.791359 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.791348    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 00:43:31.792366 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.792355    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 00:43:31.794426 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794414    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 00:43:31.794568 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794558    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 00:43:31.794616 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794587    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 00:43:31.794616 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794606    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 00:43:31.794708 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794624    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 00:43:31.794708 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794698    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 30 00:43:31.794798 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.794713    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 00:43:31.795143 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.795130    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 00:43:31.795206 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.795149    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 00:43:31.798570 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.798550    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 00:43:31.800488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.800471    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 00:43:31.800569 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.800510    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 00:43:31.802138 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802123    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 00:43:31.802138 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802143    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802153    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802161    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802174    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802183    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802195    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802204    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802214    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802222    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802234    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 00:43:31.802283 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.802242    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 00:43:31.803221 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.803211    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 00:43:31.803270 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.803223    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 00:43:31.806497 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.806480    2584 server.go:1268] "Started kubelet"
Jul 30 00:43:31.806583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.806543    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 00:43:31.806645 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.806568    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 00:43:31.806693 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.806657    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 00:43:31.807384 ip-10-0-134-28 systemd[1]: Started Kubernetes Kubelet.
Jul 30 00:43:31.807799 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.807770    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 00:43:31.807799 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.807778    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 00:43:31.808199 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.808174    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 00:43:31.808550 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.808489    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 00:43:31.809140 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.809123    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 00:43:31.812027 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.812007    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 00:43:31.815902 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.815887    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 00:43:31.815986 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.815959    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:31.816325 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.816267    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 00:43:31.817147 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.816505    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 00:43:31.817773 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.817751    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 00:43:31.817890 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.817777    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 00:43:31.817950 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.817911    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 00:43:31.817950 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.817919    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 00:43:31.817950 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.816923    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913c93b339  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.806450489 +0000 UTC m=+0.365565397,LastTimestamp:2026-07-30 00:43:31.806450489 +0000 UTC m=+0.365565397,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.818583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818566    2584 factory.go:55] Registering systemd factory
Jul 30 00:43:31.818583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818585    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 30 00:43:31.818832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818819    2584 factory.go:153] Registering CRI-O factory
Jul 30 00:43:31.818832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818834    2584 factory.go:223] Registration of the crio container factory successfully
Jul 30 00:43:31.818908 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818873    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 00:43:31.818908 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818896    2584 factory.go:103] Registering Raw factory
Jul 30 00:43:31.818908 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.818908    2584 manager.go:1196] Started watching for new ooms in manager
Jul 30 00:43:31.819225 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.819205    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 00:43:31.819225 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.819225    2584 manager.go:349] Starting recovery of all containers
Jul 30 00:43:31.819339 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.819321    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 00:43:31.831323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.831300    2584 manager.go:354] Recovery completed
Jul 30 00:43:31.835397 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.835382    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:31.838367 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838346    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:31.838452 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838379    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:31.838452 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838390    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:31.838931 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838918    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 00:43:31.838931 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838929    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 00:43:31.838999 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838945    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 00:43:31.838999 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.838929    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:31.839727 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.839711    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:31.839772 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.839741    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:31.839772 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.839756    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:31.840377 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.840287    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7ab14a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838366026 +0000 UTC m=+0.397480933,LastTimestamp:2026-07-30 00:43:31.838366026 +0000 UTC m=+0.397480933,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.840908 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.840897    2584 policy_none.go:49] "None policy: Start"
Jul 30 00:43:31.840953 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.840913    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 00:43:31.840953 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.840924    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 00:43:31.842119 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.842103    2584 policy_none.go:47] "Start"
Jul 30 00:43:31.847870 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.847804    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7af6e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838383843 +0000 UTC m=+0.397498750,LastTimestamp:2026-07-30 00:43:31.838383843 +0000 UTC m=+0.397498750,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.850489 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.850429    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7b208f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838394511 +0000 UTC m=+0.397509419,LastTimestamp:2026-07-30 00:43:31.838394511 +0000 UTC m=+0.397509419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.858510 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.858443    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-28.ec2.internal.18c6ea913e7ab14a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7ab14a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838366026 +0000 UTC m=+0.397480933,LastTimestamp:2026-07-30 00:43:31.839726257 +0000 UTC m=+0.398841168,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.869448 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.869272    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-28.ec2.internal.18c6ea913e7af6e3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7af6e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838383843 +0000 UTC m=+0.397498750,LastTimestamp:2026-07-30 00:43:31.839747626 +0000 UTC m=+0.398862536,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:31.872749 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.872732    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-72wt9"
Jul 30 00:43:31.875258 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875245    2584 manager.go:337] "Starting Device Plugin manager"
Jul 30 00:43:31.875299 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.875281    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 00:43:31.875299 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875294    2584 server.go:87] "Starting device plugin registration server"
Jul 30 00:43:31.875533 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875522    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 00:43:31.875574 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875535    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 00:43:31.875662 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875645    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 00:43:31.875783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875727    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 00:43:31.875783 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.875739    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 00:43:31.876255 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.876238    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 00:43:31.876324 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.876293    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:31.879727 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:31.879704    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-72wt9"
Jul 30 00:43:31.880380 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:31.880320    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-28.ec2.internal.18c6ea913e7b208f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-28.ec2.internal.18c6ea913e7b208f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-28.ec2.internal,UID:ip-10-0-134-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-28.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-28.ec2.internal,},FirstTimestamp:2026-07-30 00:43:31.838394511 +0000 UTC m=+0.397509419,LastTimestamp:2026-07-30 00:43:31.839761335 +0000 UTC m=+0.398876245,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-28.ec2.internal,}"
Jul 30 00:43:33.344077 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.344041    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.345238 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.345221    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.345295 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.345263    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.345295 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.345279    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.345359 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.345314    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.354277 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.354217    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.354277 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.354247    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-28.ec2.internal\": node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.373151 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.373115    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.378071 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.378045    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 00:43:33.379209 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.379192    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 00:43:33.379280 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.379231    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 00:43:33.379280 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.379250    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 00:43:33.379381 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.379286    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 00:43:33.381194 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.381174    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:33.473858 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.473810    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.479966 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.479940    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"]
Jul 30 00:43:33.480032 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.480013    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.481017 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.480998    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.481135 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.481030    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.481135 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.481040    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.482547 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.482536    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.482678 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.482648    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.482678 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.482676    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.483473 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483458    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.483549 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483479    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.483549 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483485    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.483549 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483495    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.483549 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483497    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.483549 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.483510    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.484638 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.484618    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.484738 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.484660    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.485384 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.485370    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.485445 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.485389    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.485445 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.485399    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.488211 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.488187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.488297 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.488226    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.488297 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.488251    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f6325ce07165ebf2e9979367b54cf4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-28.ec2.internal\" (UID: \"f0f6325ce07165ebf2e9979367b54cf4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.509561 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.509537    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-28.ec2.internal\" not found" node="ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.514180 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.514162    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-28.ec2.internal\" not found" node="ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.574675 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.574642    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.589270 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.589393 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.589393 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f6325ce07165ebf2e9979367b54cf4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-28.ec2.internal\" (UID: \"f0f6325ce07165ebf2e9979367b54cf4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.589393 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589334    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f6325ce07165ebf2e9979367b54cf4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-28.ec2.internal\" (UID: \"f0f6325ce07165ebf2e9979367b54cf4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.589393 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589375    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.589548 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.589416    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f8670d84f658402c20684d057cb8783-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal\" (UID: \"0f8670d84f658402c20684d057cb8783\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.675611 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.675524    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.776120 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.776085    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.812574 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.812534    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.817686 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:33.817668    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:33.876899 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.876864    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:33.977418 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:33.977354    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-28.ec2.internal\" not found"
Jul 30 00:43:34.049436 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.049412    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.084815 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.084775    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:34.092858 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.092837    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 00:43:34.094770 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.094756    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal"
Jul 30 00:43:34.104308 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.104290    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 00:43:34.126457 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.126434    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 00:43:34.126587 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.126569    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 00:43:34.126629 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.126614    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 00:43:34.163778 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.163744    2584 apiserver.go:52] "Watching apiserver"
Jul 30 00:43:34.183919 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.183891    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.184607 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.184589    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 00:43:34.184755 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.184738    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 00:43:34.185128 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.185088    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-lpxzj","openshift-dns/node-resolver-62tnh","openshift-image-registry/node-ca-xv4mc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal","openshift-multus/multus-additional-cni-plugins-xnfrk","openshift-multus/network-metrics-daemon-qb96l","openshift-network-diagnostics/network-check-target-gxgks","kube-system/konnectivity-agent-pjvpj","kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb","openshift-multus/multus-zd5pd","openshift-network-operator/iptables-alerter-27wnr","openshift-ovn-kubernetes/ovnkube-node-vdlqw","kube-system/global-pull-secret-syncer-j69sg"]
Jul 30 00:43:34.187116 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.187099    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.188507 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.188491    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.189657 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.189634    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.189788 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.189674    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lgf47\""
Jul 30 00:43:34.189788 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.189700    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 00:43:34.189788 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.189724    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.189984 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.189884    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.190743 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.190727    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 00:43:34.190835 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.190771    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.190966 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.190952    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.191041 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.191024    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 00:43:34.191041 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.191037    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8m4zf\""
Jul 30 00:43:34.191258 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.191246    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.191314 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.191296    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:34.192022 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.191987    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 00:43:34.192122 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.192036    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 00:43:34.192726 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.192705    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.192944 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.192924    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.192944 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.192938    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.193116 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.192966    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 00:43:34.193116 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.193036    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-57cbs\""
Jul 30 00:43:34.194041 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.194022    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.194132 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.194082    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:34.194982 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.194966    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 00:43:34.195224 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195209    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zhkb6\""
Jul 30 00:43:34.195274 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195240    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.195397 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195382    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.195397 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195395    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.195655 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195639    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 00:43:34.195960 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.195943    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 00:43:34.196019 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.196005    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 00:43:34.196685 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.196671    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:34.196754 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.196736    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:34.197313 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.197299    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 00:43:34.197608 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.197589    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mws9x\""
Jul 30 00:43:34.197753 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.197739    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 00:43:34.198008 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.197996    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.199753 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.199738    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.200213 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.200198    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bq2v9\""
Jul 30 00:43:34.200213 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.200208    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.200362 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.200200    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.201007 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.200989    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.201739 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.201726    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.202144 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.202127    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7hwz4\""
Jul 30 00:43:34.202219 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.202134    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.203046 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.203032    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 00:43:34.203129 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.203113    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-52xds\""
Jul 30 00:43:34.203449 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.203435    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.205827 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.205809    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 00:43:34.205907 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.205815    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:43:34.206219 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.206206    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 00:43:34.206332 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.206318    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6nwn8\""
Jul 30 00:43:34.216318 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.216299    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7724s"
Jul 30 00:43:34.224157 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.224139    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7724s"
Jul 30 00:43:34.252347 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.252296    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 00:38:33 +0000 UTC" deadline="2028-03-21 23:04:34.491028324 +0000 UTC"
Jul 30 00:43:34.252347 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.252343    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14422h21m0.238687507s"
Jul 30 00:43:34.285608 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.285588    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 00:43:34.292100 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292070    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.292231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292110    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.292231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292140    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1301c9a9-7463-4730-9548-8fc0a02803cb-serviceca\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.292231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292164    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.292231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.292231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-env-overrides\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292233    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292255    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-etc-tuned\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292278    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-registration-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292349    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-kubelet\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgrc\" (UniqueName: \"kubernetes.io/projected/9b3fb6af-f27f-4c36-ac55-d12613327f01-kube-api-access-wbgrc\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292435    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-socket-dir-parent\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.292482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-multus\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f1874a-be9b-4ee4-9aa3-07259267df2b-iptables-alerter-script\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292524    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f1874a-be9b-4ee4-9aa3-07259267df2b-host-slash\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292547    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-etc-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292570    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-os-release\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-modprobe-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292617    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-run\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292637    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-host\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292679    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-conf\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-tmp\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292762    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-script-lib\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-bin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292833    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-kubelet\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292873    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-kubernetes\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-lib-modules\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292941    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5t6\" (UniqueName: \"kubernetes.io/projected/99f1874a-be9b-4ee4-9aa3-07259267df2b-kube-api-access-sk5t6\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292966    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-var-lib-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.292988    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-ovn\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293024    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-var-lib-kubelet\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6gs\" (UniqueName: \"kubernetes.io/projected/fe532fbc-7f72-460d-b518-18f767c7a3af-kube-api-access-6t6gs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293110    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-systemd-units\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293136    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f9p9\" (UniqueName: \"kubernetes.io/projected/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-kube-api-access-5f9p9\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-kubelet-config\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293185    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-dbus\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrbm6\" (UniqueName: \"kubernetes.io/projected/e3013612-b6cd-4e9f-9497-b4a528e08842-kube-api-access-rrbm6\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293283    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-systemd\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293355    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13569e51-ed67-4de6-b024-bee5667e5212-agent-certs\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cni-binary-copy\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293461    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.293754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293492    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1301c9a9-7463-4730-9548-8fc0a02803cb-host\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293545    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-socket-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293606    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltcx8\" (UniqueName: \"kubernetes.io/projected/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-kube-api-access-ltcx8\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-k8s-cni-cncf-io\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293657    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-os-release\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-netd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-netns\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293838    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-etc-kubernetes\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-system-cni-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293917    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf88b\" (UniqueName: \"kubernetes.io/projected/86d15d74-beff-4568-9c89-660ad1fca368-kube-api-access-qf88b\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-netns\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.293980    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovn-node-metrics-cert\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-system-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.294489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294084    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cnibin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-device-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-slash\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294180    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-node-log\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294210    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13569e51-ed67-4de6-b024-bee5667e5212-konnectivity-ca\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-daemon-config\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ntx9\" (UniqueName: \"kubernetes.io/projected/1301c9a9-7463-4730-9548-8fc0a02803cb-kube-api-access-5ntx9\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-bin\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-conf-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294362    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-multus-certs\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294384    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-sys\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpmk4\" (UniqueName: \"kubernetes.io/projected/989b552b-a69a-494b-9661-926e14393953-kube-api-access-bpmk4\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294436    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-log-socket\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294484    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-hostroot\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294580    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-cnibin\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.295035 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86d15d74-beff-4568-9c89-660ad1fca368-hosts-file\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.295576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d15d74-beff-4568-9c89-660ad1fca368-tmp-dir\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.295576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysconfig\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.295576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294717    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-sys-fs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.295576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294735    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-config\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.295576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.294748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-systemd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.360358 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.360335    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.390079 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.390062    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:43:34.395024 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgrc\" (UniqueName: \"kubernetes.io/projected/9b3fb6af-f27f-4c36-ac55-d12613327f01-kube-api-access-wbgrc\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.395113 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395037    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-socket-dir-parent\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395113 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395061    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-multus\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395113 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f1874a-be9b-4ee4-9aa3-07259267df2b-iptables-alerter-script\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395125    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-multus\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395128    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f1874a-be9b-4ee4-9aa3-07259267df2b-host-slash\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395166    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f1874a-be9b-4ee4-9aa3-07259267df2b-host-slash\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395165    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-socket-dir-parent\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395188    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-etc-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395219    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-etc-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.395256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-os-release\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-modprobe-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-run\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-os-release\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395320    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-host\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395359    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-conf\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-tmp\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395386    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-host\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395390    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-script-lib\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395414    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-bin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-kubelet\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-modprobe-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-kubernetes\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-lib-modules\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5t6\" (UniqueName: \"kubernetes.io/projected/99f1874a-be9b-4ee4-9aa3-07259267df2b-kube-api-access-sk5t6\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395478    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-run\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395492    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-var-lib-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.395582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-ovn\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-conf\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395549    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395559    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-var-lib-kubelet\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395588    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6gs\" (UniqueName: \"kubernetes.io/projected/fe532fbc-7f72-460d-b518-18f767c7a3af-kube-api-access-6t6gs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-systemd-units\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395655    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-lib-modules\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-cni-bin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-var-lib-kubelet\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395706    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395740    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-systemd-units\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-var-lib-kubelet\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395813    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-kubernetes\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395783    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f1874a-be9b-4ee4-9aa3-07259267df2b-iptables-alerter-script\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f9p9\" (UniqueName: \"kubernetes.io/projected/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-kube-api-access-5f9p9\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-kubelet-config\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-dbus\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.396439 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395937    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-var-lib-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-ovn\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396002    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-kubelet-config\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.395943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrbm6\" (UniqueName: \"kubernetes.io/projected/e3013612-b6cd-4e9f-9497-b4a528e08842-kube-api-access-rrbm6\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396022    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-script-lib\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-systemd\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396100    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-systemd\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13569e51-ed67-4de6-b024-bee5667e5212-agent-certs\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396132    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cni-binary-copy\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396147    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396153    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1301c9a9-7463-4730-9548-8fc0a02803cb-host\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-dbus\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396201    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396227    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-socket-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltcx8\" (UniqueName: \"kubernetes.io/projected/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-kube-api-access-ltcx8\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.397312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396260    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1301c9a9-7463-4730-9548-8fc0a02803cb-host\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396276    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-k8s-cni-cncf-io\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysctl-d\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396301    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-os-release\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396358    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-netd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-netns\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-etc-kubernetes\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-system-cni-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396461    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396450    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396488    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf88b\" (UniqueName: \"kubernetes.io/projected/86d15d74-beff-4568-9c89-660ad1fca368-kube-api-access-qf88b\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-netns\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396535    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovn-node-metrics-cert\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396559    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-system-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396561    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-netns\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.398151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396612    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-etc-kubernetes\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396612    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cni-binary-copy\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cnibin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396664    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-cnibin\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-device-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396664    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-system-cni-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-slash\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-netns\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396737    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-os-release\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396739    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-slash\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-node-log\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396863    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13569e51-ed67-4de6-b024-bee5667e5212-konnectivity-ca\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396892    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-daemon-config\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ntx9\" (UniqueName: \"kubernetes.io/projected/1301c9a9-7463-4730-9548-8fc0a02803cb-kube-api-access-5ntx9\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396946    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-bin\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396969    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-conf-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.396994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-multus-certs\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-sys\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.399030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397038    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmk4\" (UniqueName: \"kubernetes.io/projected/989b552b-a69a-494b-9661-926e14393953-kube-api-access-bpmk4\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397069    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-log-socket\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397094    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-hostroot\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397141    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-cnibin\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397161    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86d15d74-beff-4568-9c89-660ad1fca368-hosts-file\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397185    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d15d74-beff-4568-9c89-660ad1fca368-tmp-dir\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397190    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-socket-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397211    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysconfig\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-sys-fs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397238    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-k8s-cni-cncf-io\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-system-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-config\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3013612-b6cd-4e9f-9497-b4a528e08842-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-systemd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-cni-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.399711 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397326    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-systemd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397342    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-netd\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397349    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-log-socket\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397377    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1301c9a9-7463-4730-9548-8fc0a02803cb-serviceca\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397382    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-run-openvswitch\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397411    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397464    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-env-overrides\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-node-log\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397496    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397516    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-etc-tuned\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-run-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397541    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-registration-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-kubelet\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397586    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-registration-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.400244 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.397688    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.397764    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:34.897723345 +0000 UTC m=+2.088671673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397828    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1301c9a9-7463-4730-9548-8fc0a02803cb-serviceca\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397886    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-hostroot\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397415    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.397936    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3013612-b6cd-4e9f-9497-b4a528e08842-cnibin\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.397999    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398001    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-daemon-config\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.398046    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:43:34.898029975 +0000 UTC m=+2.088978299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-device-dir\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398291    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-cni-bin\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398332    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-multus-conf-dir\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398368    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-host-run-multus-certs\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-sys\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398438    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13569e51-ed67-4de6-b024-bee5667e5212-konnectivity-ca\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/989b552b-a69a-494b-9661-926e14393953-etc-sysconfig\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-env-overrides\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.400746 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398486    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe532fbc-7f72-460d-b518-18f767c7a3af-sys-fs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398536    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-host-kubelet\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86d15d74-beff-4568-9c89-660ad1fca368-hosts-file\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.398730    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d15d74-beff-4568-9c89-660ad1fca368-tmp-dir\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.399370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovn-node-metrics-cert\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.399388    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-tmp\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.399497    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-ovnkube-config\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.399542    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13569e51-ed67-4de6-b024-bee5667e5212-agent-certs\") pod \"konnectivity-agent-pjvpj\" (UID: \"13569e51-ed67-4de6-b024-bee5667e5212\") " pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.401247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.399681    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/989b552b-a69a-494b-9661-926e14393953-etc-tuned\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.403851 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.403821    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5t6\" (UniqueName: \"kubernetes.io/projected/99f1874a-be9b-4ee4-9aa3-07259267df2b-kube-api-access-sk5t6\") pod \"iptables-alerter-27wnr\" (UID: \"99f1874a-be9b-4ee4-9aa3-07259267df2b\") " pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.403980 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.403960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6gs\" (UniqueName: \"kubernetes.io/projected/fe532fbc-7f72-460d-b518-18f767c7a3af-kube-api-access-6t6gs\") pod \"aws-ebs-csi-driver-node-nd4nb\" (UID: \"fe532fbc-7f72-460d-b518-18f767c7a3af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.403980 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.403971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgrc\" (UniqueName: \"kubernetes.io/projected/9b3fb6af-f27f-4c36-ac55-d12613327f01-kube-api-access-wbgrc\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.404605 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.404586    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f9p9\" (UniqueName: \"kubernetes.io/projected/45751b95-3ef0-4dd9-9a0c-75ee1f348f2f-kube-api-access-5f9p9\") pod \"multus-zd5pd\" (UID: \"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f\") " pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.407359 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.407343    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:34.407464 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.407364    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:34.407464 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.407376    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:34.407464 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.407451    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:34.907434082 +0000 UTC m=+2.098382409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:34.409735 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.409704    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltcx8\" (UniqueName: \"kubernetes.io/projected/5fdd4d23-4abd-4f36-bd82-9315ff9acc12-kube-api-access-ltcx8\") pod \"ovnkube-node-vdlqw\" (UID: \"5fdd4d23-4abd-4f36-bd82-9315ff9acc12\") " pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.409975 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.409953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf88b\" (UniqueName: \"kubernetes.io/projected/86d15d74-beff-4568-9c89-660ad1fca368-kube-api-access-qf88b\") pod \"node-resolver-62tnh\" (UID: \"86d15d74-beff-4568-9c89-660ad1fca368\") " pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.410229 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.410210    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ntx9\" (UniqueName: \"kubernetes.io/projected/1301c9a9-7463-4730-9548-8fc0a02803cb-kube-api-access-5ntx9\") pod \"node-ca-xv4mc\" (UID: \"1301c9a9-7463-4730-9548-8fc0a02803cb\") " pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.410272 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.410218    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrbm6\" (UniqueName: \"kubernetes.io/projected/e3013612-b6cd-4e9f-9497-b4a528e08842-kube-api-access-rrbm6\") pod \"multus-additional-cni-plugins-xnfrk\" (UID: \"e3013612-b6cd-4e9f-9497-b4a528e08842\") " pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.410438 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.410424    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmk4\" (UniqueName: \"kubernetes.io/projected/989b552b-a69a-494b-9661-926e14393953-kube-api-access-bpmk4\") pod \"tuned-lpxzj\" (UID: \"989b552b-a69a-494b-9661-926e14393953\") " pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.518583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.518510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb"
Jul 30 00:43:34.522766 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.522724    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8670d84f658402c20684d057cb8783.slice/crio-2562b03a8f8eb6baa6e4f083577d7ff353911e4477e6fcacbf51f3131e3b9f63 WatchSource:0}: Error finding container 2562b03a8f8eb6baa6e4f083577d7ff353911e4477e6fcacbf51f3131e3b9f63: Status 404 returned error can't find the container with id 2562b03a8f8eb6baa6e4f083577d7ff353911e4477e6fcacbf51f3131e3b9f63
Jul 30 00:43:34.526296 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.526275    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe532fbc_7f72_460d_b518_18f767c7a3af.slice/crio-4315d5c77af5f212c4fb159e626732690127d4e7e2eccb7e5d216a734f3986be WatchSource:0}: Error finding container 4315d5c77af5f212c4fb159e626732690127d4e7e2eccb7e5d216a734f3986be: Status 404 returned error can't find the container with id 4315d5c77af5f212c4fb159e626732690127d4e7e2eccb7e5d216a734f3986be
Jul 30 00:43:34.535109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.535084    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:34.542437 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.542415    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdd4d23_4abd_4f36_bd82_9315ff9acc12.slice/crio-402f36925909d0c080e244118720ed466b0970795687aa247b4be4c159bcb2a1 WatchSource:0}: Error finding container 402f36925909d0c080e244118720ed466b0970795687aa247b4be4c159bcb2a1: Status 404 returned error can't find the container with id 402f36925909d0c080e244118720ed466b0970795687aa247b4be4c159bcb2a1
Jul 30 00:43:34.556495 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.556473    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xv4mc"
Jul 30 00:43:34.565186 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.565161    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1301c9a9_7463_4730_9548_8fc0a02803cb.slice/crio-4f49be1d41fae320989b21c6b1fcc144ba7f5a5c0ac4a5fe637dd260ad32395c WatchSource:0}: Error finding container 4f49be1d41fae320989b21c6b1fcc144ba7f5a5c0ac4a5fe637dd260ad32395c: Status 404 returned error can't find the container with id 4f49be1d41fae320989b21c6b1fcc144ba7f5a5c0ac4a5fe637dd260ad32395c
Jul 30 00:43:34.577885 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.577868    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnfrk"
Jul 30 00:43:34.584531 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.584507    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3013612_b6cd_4e9f_9497_b4a528e08842.slice/crio-f9846a0b63a55332e750aff7e514a026bcafb5b1f9b6269fd064ae6c53b58c2e WatchSource:0}: Error finding container f9846a0b63a55332e750aff7e514a026bcafb5b1f9b6269fd064ae6c53b58c2e: Status 404 returned error can't find the container with id f9846a0b63a55332e750aff7e514a026bcafb5b1f9b6269fd064ae6c53b58c2e
Jul 30 00:43:34.592890 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.592869    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:34.601492 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.601472    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13569e51_ed67_4de6_b024_bee5667e5212.slice/crio-c0ee4472ca59eb53eb12d37f2b94f013a33cbe58748b84ed16c0dd9f8738ea48 WatchSource:0}: Error finding container c0ee4472ca59eb53eb12d37f2b94f013a33cbe58748b84ed16c0dd9f8738ea48: Status 404 returned error can't find the container with id c0ee4472ca59eb53eb12d37f2b94f013a33cbe58748b84ed16c0dd9f8738ea48
Jul 30 00:43:34.611486 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.611468    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-62tnh"
Jul 30 00:43:34.618317 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.618289    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d15d74_beff_4568_9c89_660ad1fca368.slice/crio-9e0c2227181136ae146b2de5095141922b60f77bdc786fceaf6a673ace2e969c WatchSource:0}: Error finding container 9e0c2227181136ae146b2de5095141922b60f77bdc786fceaf6a673ace2e969c: Status 404 returned error can't find the container with id 9e0c2227181136ae146b2de5095141922b60f77bdc786fceaf6a673ace2e969c
Jul 30 00:43:34.629666 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.629642    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpxzj"
Jul 30 00:43:34.636932 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.636911    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989b552b_a69a_494b_9661_926e14393953.slice/crio-e464978249d6ae83b2882b8fa1cad2dea5e261e5355734862a4920cab59fbffb WatchSource:0}: Error finding container e464978249d6ae83b2882b8fa1cad2dea5e261e5355734862a4920cab59fbffb: Status 404 returned error can't find the container with id e464978249d6ae83b2882b8fa1cad2dea5e261e5355734862a4920cab59fbffb
Jul 30 00:43:34.645957 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.645940    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zd5pd"
Jul 30 00:43:34.652003 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.651976    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-27wnr"
Jul 30 00:43:34.652818 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.652782    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45751b95_3ef0_4dd9_9a0c_75ee1f348f2f.slice/crio-72002207b8cd29284a8389c66640450c7462649cc917741f8e00ca93cbdac278 WatchSource:0}: Error finding container 72002207b8cd29284a8389c66640450c7462649cc917741f8e00ca93cbdac278: Status 404 returned error can't find the container with id 72002207b8cd29284a8389c66640450c7462649cc917741f8e00ca93cbdac278
Jul 30 00:43:34.660370 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:43:34.660347    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f1874a_be9b_4ee4_9aa3_07259267df2b.slice/crio-8cb5e75ddfaf680f9ce344a4af15e8cf5d3aae27197f684ff526c4183e3989ba WatchSource:0}: Error finding container 8cb5e75ddfaf680f9ce344a4af15e8cf5d3aae27197f684ff526c4183e3989ba: Status 404 returned error can't find the container with id 8cb5e75ddfaf680f9ce344a4af15e8cf5d3aae27197f684ff526c4183e3989ba
Jul 30 00:43:34.902018 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.901941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:34.902018 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:34.901975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:34.902232 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.902093    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:34.902232 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.902149    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.902130814 +0000 UTC m=+3.093079137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:34.902232 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.902095    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:34.902388 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:34.902250    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.902226956 +0000 UTC m=+3.093175279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.002659 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.002620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:35.002867 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.002777    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:35.002867 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.002811    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:35.002867 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.002825    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:35.003050 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.002882    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:36.002865041 +0000 UTC m=+3.193813353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:35.226733 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.226589    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:34 +0000 UTC" deadline="2028-05-07 16:30:11.326750492 +0000 UTC"
Jul 30 00:43:35.226733 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.226661    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15543h46m36.100094495s"
Jul 30 00:43:35.400555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.400525    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:35.400715 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.400658    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:35.401445 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.401148    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:35.401445 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.401247    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:35.401445 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.401308    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:35.401445 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.401372    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:35.420218 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.420096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpxzj" event={"ID":"989b552b-a69a-494b-9661-926e14393953","Type":"ContainerStarted","Data":"e464978249d6ae83b2882b8fa1cad2dea5e261e5355734862a4920cab59fbffb"}
Jul 30 00:43:35.436085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.436048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjvpj" event={"ID":"13569e51-ed67-4de6-b024-bee5667e5212","Type":"ContainerStarted","Data":"c0ee4472ca59eb53eb12d37f2b94f013a33cbe58748b84ed16c0dd9f8738ea48"}
Jul 30 00:43:35.443214 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.443017    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:35.447127 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.447028    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerStarted","Data":"f9846a0b63a55332e750aff7e514a026bcafb5b1f9b6269fd064ae6c53b58c2e"}
Jul 30 00:43:35.466812 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.466331    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xv4mc" event={"ID":"1301c9a9-7463-4730-9548-8fc0a02803cb","Type":"ContainerStarted","Data":"4f49be1d41fae320989b21c6b1fcc144ba7f5a5c0ac4a5fe637dd260ad32395c"}
Jul 30 00:43:35.475609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.475571    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal" event={"ID":"0f8670d84f658402c20684d057cb8783","Type":"ContainerStarted","Data":"2562b03a8f8eb6baa6e4f083577d7ff353911e4477e6fcacbf51f3131e3b9f63"}
Jul 30 00:43:35.493604 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.493519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal" event={"ID":"f0f6325ce07165ebf2e9979367b54cf4","Type":"ContainerStarted","Data":"9057ddab3d51db11323098a8447e8030af3c51a71afa42fc1caf3dcfdf53d7db"}
Jul 30 00:43:35.507896 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.507855    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-27wnr" event={"ID":"99f1874a-be9b-4ee4-9aa3-07259267df2b","Type":"ContainerStarted","Data":"8cb5e75ddfaf680f9ce344a4af15e8cf5d3aae27197f684ff526c4183e3989ba"}
Jul 30 00:43:35.528085 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.528048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zd5pd" event={"ID":"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f","Type":"ContainerStarted","Data":"72002207b8cd29284a8389c66640450c7462649cc917741f8e00ca93cbdac278"}
Jul 30 00:43:35.537670 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.537631    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-62tnh" event={"ID":"86d15d74-beff-4568-9c89-660ad1fca368","Type":"ContainerStarted","Data":"9e0c2227181136ae146b2de5095141922b60f77bdc786fceaf6a673ace2e969c"}
Jul 30 00:43:35.551245 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.551198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"402f36925909d0c080e244118720ed466b0970795687aa247b4be4c159bcb2a1"}
Jul 30 00:43:35.574991 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.574957    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb" event={"ID":"fe532fbc-7f72-460d-b518-18f767c7a3af","Type":"ContainerStarted","Data":"4315d5c77af5f212c4fb159e626732690127d4e7e2eccb7e5d216a734f3986be"}
Jul 30 00:43:35.913409 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.913322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:35.913409 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:35.913379    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:35.913639 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.913506    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.913639 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.913567    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:37.9135497 +0000 UTC m=+5.104498022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.913986 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.913968    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.914062 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:35.914019    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:43:37.914005299 +0000 UTC m=+5.104953623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:36.014571 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:36.014532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:36.014809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:36.014765    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:36.014809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:36.014807    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:36.014966 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:36.014821    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:36.014966 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:36.014879    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:38.014861567 +0000 UTC m=+5.205809893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:36.227497 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:36.227353    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:34 +0000 UTC" deadline="2028-04-11 08:08:32.573409804 +0000 UTC"
Jul 30 00:43:36.227497 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:36.227436    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14911h24m56.345978335s"
Jul 30 00:43:36.343736 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:36.343442    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:37.388973 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:37.388935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:37.389419 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.389074    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:37.389495 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:37.389458    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:37.389495 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:37.389479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:37.389586 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.389552    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:37.389881 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.389679    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:37.934609    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:37.934660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.934815    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.934882    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:41.934861764 +0000 UTC m=+9.125810095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.935262    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:37.935364 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:37.935314    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:43:41.935297549 +0000 UTC m=+9.126245874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:38.035639 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:38.035603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:38.035896 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:38.035819    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:38.035896 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:38.035843    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:38.035896 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:38.035856    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:38.036065 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:38.035922    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:42.035903039 +0000 UTC m=+9.226851368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:39.380632 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:39.380480    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:39.381117 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:39.380643    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:39.381117 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:39.380728    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:39.381232 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:39.381137    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:39.381284 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:39.381231    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:39.381284 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:39.381269    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:41.380443 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:41.380407    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:41.380830 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.380557    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:41.380830 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:41.380676    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:41.380830 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.380786    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:41.380946 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:41.380883    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:41.380979 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.380957    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:41.972043 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:41.972001    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:41.972043 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:41.972044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:41.972276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.972196    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:41.972276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.972257    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:49.972237501 +0000 UTC m=+17.163185830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:41.972399 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.972329    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:41.972399 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:41.972366    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:43:49.97235475 +0000 UTC m=+17.163303075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:42.073320 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:42.073233    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:42.073494 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:42.073431    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:42.073494 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:42.073460    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:42.073494 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:42.073475    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:42.073653 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:42.073539    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:50.073517836 +0000 UTC m=+17.264466165 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:43.380274 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:43.380243    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:43.380733 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:43.380372    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:43.381849 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:43.381690    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:43.381849 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:43.381807    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:43.382034 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:43.381983    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:43.382102 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:43.382082    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:45.379718 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:45.379681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:45.380198 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:45.379733    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:45.380198 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:45.379681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:45.380198 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:45.379854    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:45.380198 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:45.379966    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:45.380198 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:45.380105    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:47.379555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:47.379516    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:47.379969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:47.379519    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:47.379969 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:47.379650    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:47.379969 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:47.379516    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:47.379969 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:47.379727    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:47.379969 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:47.379820    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:49.380551 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:49.380503    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:49.380551 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:49.380540    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:49.381075 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:49.380506    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:49.381075 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:49.380619    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:49.381075 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:49.380715    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:49.381075 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:49.380850    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:50.037869 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:50.037824    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:50.038117 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:50.037917    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:50.038117 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.037994    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:50.038117 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.038025    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:50.038117 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.038070    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.038047883 +0000 UTC m=+33.228996215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:50.038117 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.038089    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.038081329 +0000 UTC m=+33.229029638 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:50.138749 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:50.138702    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:50.138936 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.138879    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:50.138936 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.138903    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:50.138936 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.138913    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:50.139095 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:50.138996    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.138964058 +0000 UTC m=+33.329912372 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:51.380333 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:51.380294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:51.380776 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:51.380294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:51.380776 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:51.380445    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:51.380776 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:51.380295    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:51.380776 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:51.380518    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:51.380776 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:51.380620    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:53.380202 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:53.380167    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:53.380619 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:53.380278    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:53.380619 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:53.380306    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:53.380619 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:53.380277    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:53.380619 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:53.380377    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:53.380619 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:53.380482    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:54.613920 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.613562    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-62tnh" event={"ID":"86d15d74-beff-4568-9c89-660ad1fca368","Type":"ContainerStarted","Data":"b59e142b131fdc6392a7dc1ca10f353d2a5d4d73d8f6f22cb7224db35bca8725"}
Jul 30 00:43:54.615690 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.615665    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"926b60e0e78b8884effb841bb87966c6ca5ea5e66c629e6f69cdaccfcabcbb2f"}
Jul 30 00:43:54.615864 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.615726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"c70d6958a803fcc443f1183e96250d522410c818b5969c9df4f204b242cd60fe"}
Jul 30 00:43:54.615864 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.615764    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"f1a1c8503f5d1ff7318eee7be451a6dd17161d948ad624972ee65c8b727108d1"}
Jul 30 00:43:54.617108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.617083    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb" event={"ID":"fe532fbc-7f72-460d-b518-18f767c7a3af","Type":"ContainerStarted","Data":"4d2afd511e0aca93279bdda211047a966ee28fac90a49bb5984e40bea6a3aff2"}
Jul 30 00:43:54.618389 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.618372    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpxzj" event={"ID":"989b552b-a69a-494b-9661-926e14393953","Type":"ContainerStarted","Data":"d9309e341a70b7938bbf450bfe3e8ba034f64eab8f5d06031eb49fd4301635a1"}
Jul 30 00:43:54.619616 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.619597    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjvpj" event={"ID":"13569e51-ed67-4de6-b024-bee5667e5212","Type":"ContainerStarted","Data":"54982994529dfbf2e0eb6c072382d9b4f966d2189f1b215279f1299bb7b820e5"}
Jul 30 00:43:54.620946 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.620928    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="d1f0bcf16667d723356594b7d682583e7a437d100d996eb8547191497955005f" exitCode=0
Jul 30 00:43:54.621018 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.620994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"d1f0bcf16667d723356594b7d682583e7a437d100d996eb8547191497955005f"}
Jul 30 00:43:54.622296 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.622276    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xv4mc" event={"ID":"1301c9a9-7463-4730-9548-8fc0a02803cb","Type":"ContainerStarted","Data":"dcb96ac837207ca37ed115b0afd04457cee305c0c0fd55d6e63ece3d2cfb647a"}
Jul 30 00:43:54.623823 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.623789    2584 generic.go:356] "Generic (PLEG): container finished" podID="0f8670d84f658402c20684d057cb8783" containerID="6669ddc9323f993bfc3bd547bb216e5fe21769b302110b3e2103fe750a42d1ee" exitCode=0
Jul 30 00:43:54.623908 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.623862    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal" event={"ID":"0f8670d84f658402c20684d057cb8783","Type":"ContainerDied","Data":"6669ddc9323f993bfc3bd547bb216e5fe21769b302110b3e2103fe750a42d1ee"}
Jul 30 00:43:54.625189 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.625164    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal" event={"ID":"f0f6325ce07165ebf2e9979367b54cf4","Type":"ContainerStarted","Data":"928894cdce01a7d094bb485c4f6244adf800eeb097f36d1651d89a2004ced862"}
Jul 30 00:43:54.626513 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.626482    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zd5pd" event={"ID":"45751b95-3ef0-4dd9-9a0c-75ee1f348f2f","Type":"ContainerStarted","Data":"db23f2309fdff82e3a53553bfd8d75bf850c19f6cbb2dc99897071b049698de7"}
Jul 30 00:43:54.629464 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.629430    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-62tnh" podStartSLOduration=2.171853165 podStartE2EDuration="21.62941995s" totalImagesPullingTime="19.457566785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.62892678 +0000 UTC m=+21.819875111" watchObservedRunningTime="2026-07-30 00:43:54.62941995 +0000 UTC m=+21.820368282"
Jul 30 00:43:54.658660 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.658620    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lpxzj" podStartSLOduration=2.219685031 podStartE2EDuration="21.658606403s" totalImagesPullingTime="19.438921372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.644302141 +0000 UTC m=+21.835250501" watchObservedRunningTime="2026-07-30 00:43:54.658606403 +0000 UTC m=+21.849554727"
Jul 30 00:43:54.659006 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.658984    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-28.ec2.internal" podStartSLOduration=20.658978714 podStartE2EDuration="20.658978714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.658678965 +0000 UTC m=+21.849627296" watchObservedRunningTime="2026-07-30 00:43:54.658978714 +0000 UTC m=+21.849927073"
Jul 30 00:43:54.677327 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.677291    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xv4mc" podStartSLOduration=2.145684039 podStartE2EDuration="21.677279543s" totalImagesPullingTime="19.531595504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.677250714 +0000 UTC m=+21.868199045" watchObservedRunningTime="2026-07-30 00:43:54.677279543 +0000 UTC m=+21.868227873"
Jul 30 00:43:54.713165 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.713132    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pjvpj" podStartSLOduration=2.238742612 podStartE2EDuration="21.713119542s" totalImagesPullingTime="19.47437693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.712764514 +0000 UTC m=+21.903712847" watchObservedRunningTime="2026-07-30 00:43:54.713119542 +0000 UTC m=+21.904067881"
Jul 30 00:43:54.730707 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:54.730678    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zd5pd" podStartSLOduration=2.187028168 podStartE2EDuration="21.730666525s" totalImagesPullingTime="19.543638357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:54.730512026 +0000 UTC m=+21.921460356" watchObservedRunningTime="2026-07-30 00:43:54.730666525 +0000 UTC m=+21.921614857"
Jul 30 00:43:55.032761 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.032726    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:55.033585 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.033556    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:43:55.380488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.380166    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:55.380654 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.380243    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:55.380654 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:55.380531    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:55.380654 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.380349    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:55.380654 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:55.380614    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:55.380885 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:55.380661    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:55.634484 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.634412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"102daca726deee450c4bc96a320dd0d7bdc8e2d3cb3b093581769cd471eb190d"}
Jul 30 00:43:55.634893 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.634540    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"4d51f53580ebed9eb1392f3f2fdb17fb85c64fc72a430c146b1f7ca47a7db9a8"}
Jul 30 00:43:55.634893 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.634591    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"65046bd045991cff3ebc3acd1d5ad64200c4f929d51dafba3e9a98fcbe0e42ca"}
Jul 30 00:43:55.821936 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:55.821912    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 00:43:56.255396 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.255280    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T00:43:55.821930098Z","UUID":"3f80831c-f0f4-49a4-814d-56208ce47935","Handler":null,"Name":"","Endpoint":""}
Jul 30 00:43:56.257055 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.257023    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 00:43:56.257183 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.257063    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 00:43:56.638356 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.638279    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb" event={"ID":"fe532fbc-7f72-460d-b518-18f767c7a3af","Type":"ContainerStarted","Data":"44bb377811d119b6e8763fd5ed96dffa0c6ab56ea83c372b6cce0029ba65c073"}
Jul 30 00:43:56.639951 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.639928    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal" event={"ID":"0f8670d84f658402c20684d057cb8783","Type":"ContainerStarted","Data":"f33ae03fd4be7adaedad6ce2057235b2a65e12d7eba9617895b316ddca790186"}
Jul 30 00:43:56.641146 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.641119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-27wnr" event={"ID":"99f1874a-be9b-4ee4-9aa3-07259267df2b","Type":"ContainerStarted","Data":"6315a81d7c1e103b4d554c72bb925741bdaa8b08ca6ac9f99b50ce5701549d77"}
Jul 30 00:43:56.641146 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.641138    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 00:43:56.658499 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:56.658442    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-28.ec2.internal" podStartSLOduration=22.658423928 podStartE2EDuration="22.658423928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:56.658182059 +0000 UTC m=+23.849130434" watchObservedRunningTime="2026-07-30 00:43:56.658423928 +0000 UTC m=+23.849372262"
Jul 30 00:43:57.379893 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.379864    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:57.380140 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.379868    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:57.380140 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:57.379982    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:57.380140 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.379868    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:57.380140 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:57.380046    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:57.380140 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:57.380117    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:57.646050 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.645956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb" event={"ID":"fe532fbc-7f72-460d-b518-18f767c7a3af","Type":"ContainerStarted","Data":"f91aadf72112c57be4a5647e9c2363aae2b4ca8530c04d38dd378e7b138548de"}
Jul 30 00:43:57.651579 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.651544    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"d5a6c051ff57d42af287b84da8e24b78a66dc62ec4d86fefa4b238b9d164a1d3"}
Jul 30 00:43:57.662575 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.662528    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd4nb" podStartSLOduration=2.91361378 podStartE2EDuration="24.662516052s" totalImagesPullingTime="21.748902272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:57.662329153 +0000 UTC m=+24.853277484" watchObservedRunningTime="2026-07-30 00:43:57.662516052 +0000 UTC m=+24.853464382"
Jul 30 00:43:57.662748 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:57.662722    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-27wnr" podStartSLOduration=5.214158969 podStartE2EDuration="24.662714688s" totalImagesPullingTime="19.448555719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:56.675517975 +0000 UTC m=+23.866466305" watchObservedRunningTime="2026-07-30 00:43:57.662714688 +0000 UTC m=+24.853663018"
Jul 30 00:43:59.380229 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.379997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:43:59.380229 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.379997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:43:59.380750 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:59.380249    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:43:59.380750 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.380013    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:43:59.380750 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:59.380338    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:43:59.380750 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:43:59.380420    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:43:59.658931 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.658853    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" event={"ID":"5fdd4d23-4abd-4f36-bd82-9315ff9acc12","Type":"ContainerStarted","Data":"4423ab65049915158ec01abc6172f6e60c1b95a837845d4efa6634e14bf65859"}
Jul 30 00:43:59.659171 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.659153    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:59.659278 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.659224    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:59.674484 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.674453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:59.674759 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.674745    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:43:59.684880 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.684835    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw" podStartSLOduration=6.974114468 podStartE2EDuration="26.684820601s" totalImagesPullingTime="19.710706133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:59.684264859 +0000 UTC m=+26.875213189" watchObservedRunningTime="2026-07-30 00:43:59.684820601 +0000 UTC m=+26.875768929"
Jul 30 00:43:59.889242 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:43:59.889210    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:44:01.152362 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.152280    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qb96l"]
Jul 30 00:44:01.153225 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.153201    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j69sg"]
Jul 30 00:44:01.153395 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.153320    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:01.153477 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:01.153420    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:44:01.153781 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.153756    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxgks"]
Jul 30 00:44:01.153887 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.153878    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:01.154000 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:01.153973    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:44:01.154000 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:01.154010    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:01.154151 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:01.154097    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:44:03.380950 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:03.380749    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:03.381382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:03.380837    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:03.381382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:03.380852    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:03.381382 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:03.381127    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:44:03.381382 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:03.381133    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:44:03.381382 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:03.381011    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:44:03.670314 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:03.670266    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="ba3746cf62a152a9c44125a451ce36b0f51dc6728a7368065cfe177a30763cfa" exitCode=0
Jul 30 00:44:03.670488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:03.670334    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"ba3746cf62a152a9c44125a451ce36b0f51dc6728a7368065cfe177a30763cfa"}
Jul 30 00:44:04.126408 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:04.126370    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:44:04.126602 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:04.126518    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 00:44:04.126990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:04.126969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pjvpj"
Jul 30 00:44:04.674202 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:04.674125    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="3c7c7fbeed6792f67f42660b2a22ebbff8fbb6102c8a714124f5aaab1e2f520e" exitCode=0
Jul 30 00:44:04.674610 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:04.674215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"3c7c7fbeed6792f67f42660b2a22ebbff8fbb6102c8a714124f5aaab1e2f520e"}
Jul 30 00:44:05.379685 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:05.379652    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:05.379866 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:05.379652    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:05.379866 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:05.379767    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb96l" podUID="9b3fb6af-f27f-4c36-ac55-d12613327f01"
Jul 30 00:44:05.379953 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:05.379867    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j69sg" podUID="b10cef9e-bbe3-45a5-aefd-30c11fb84b4d"
Jul 30 00:44:05.379953 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:05.379665    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:05.380015 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:05.379966    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxgks" podUID="cd8796e7-7571-4ead-8b4e-bc6dbe502ca4"
Jul 30 00:44:05.678742 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:05.678667    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="503bc30be8b559cf49fcf484881fd1102773f9ceab06e915118e7f88c66cf4f5" exitCode=0
Jul 30 00:44:05.678742 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:05.678720    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"503bc30be8b559cf49fcf484881fd1102773f9ceab06e915118e7f88c66cf4f5"}
Jul 30 00:44:06.061411 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:06.061375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:06.061582 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:06.061422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:06.061582 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.061533    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:44:06.061673 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.061591    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs podName:9b3fb6af-f27f-4c36-ac55-d12613327f01 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:38.061572439 +0000 UTC m=+65.252520764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs") pod "network-metrics-daemon-qb96l" (UID: "9b3fb6af-f27f-4c36-ac55-d12613327f01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:44:06.061673 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.061534    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:44:06.061781 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.061678    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret podName:b10cef9e-bbe3-45a5-aefd-30c11fb84b4d nodeName:}" failed. No retries permitted until 2026-07-30 00:44:38.061658803 +0000 UTC m=+65.252607115 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret") pod "global-pull-secret-syncer-j69sg" (UID: "b10cef9e-bbe3-45a5-aefd-30c11fb84b4d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:44:06.161982 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:06.161947    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:06.162146 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.162124    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:44:06.162191 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.162148    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:44:06.162191 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.162162    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqs5q for pod openshift-network-diagnostics/network-check-target-gxgks: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:44:06.162257 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:06.162223    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q podName:cd8796e7-7571-4ead-8b4e-bc6dbe502ca4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:38.162205142 +0000 UTC m=+65.353153451 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqs5q" (UniqueName: "kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q") pod "network-check-target-gxgks" (UID: "cd8796e7-7571-4ead-8b4e-bc6dbe502ca4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:44:07.163497 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.163473    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-28.ec2.internal" event="NodeReady"
Jul 30 00:44:07.163946 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.163618    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 00:44:07.206517 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.206484    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t69pf"]
Jul 30 00:44:07.208843 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.208816    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nxzv5"]
Jul 30 00:44:07.208983 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.208921    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.210702 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.210683    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.211234 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.211213    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 00:44:07.211342 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.211322    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\""
Jul 30 00:44:07.211404 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.211324    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 00:44:07.212807 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.212762    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mgwl2\""
Jul 30 00:44:07.212923 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.212849    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 00:44:07.212995 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.212959    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 00:44:07.213226 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.213205    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 00:44:07.220069 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.220050    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxzv5"]
Jul 30 00:44:07.222872 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.222813    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t69pf"]
Jul 30 00:44:07.370583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-config-volume\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.370777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krlf5\" (UniqueName: \"kubernetes.io/projected/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-kube-api-access-krlf5\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.370777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-tmp-dir\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.370777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwvj6\" (UniqueName: \"kubernetes.io/projected/0f37938d-32a7-444e-b325-ea4e94f96f31-kube-api-access-fwvj6\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.370777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.370777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.370767    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f37938d-32a7-444e-b325-ea4e94f96f31-cert\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.379706 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.379681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:07.379855 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.379714    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:07.379855 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.379681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:07.382496 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382472    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\""
Jul 30 00:44:07.382603 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382527    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 00:44:07.382768 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382752    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 00:44:07.382850 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382767    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 00:44:07.382909 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382852    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\""
Jul 30 00:44:07.382960 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.382930    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 00:44:07.472058 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.471970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krlf5\" (UniqueName: \"kubernetes.io/projected/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-kube-api-access-krlf5\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.472058 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472013    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-tmp-dir\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.472058 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwvj6\" (UniqueName: \"kubernetes.io/projected/0f37938d-32a7-444e-b325-ea4e94f96f31-kube-api-access-fwvj6\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.472305 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472103    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.472305 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f37938d-32a7-444e-b325-ea4e94f96f31-cert\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.472305 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472174    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-config-volume\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.472447 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:07.472366    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:07.472447 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472400    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-tmp-dir\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.472447 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:07.472439    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:07.972417482 +0000 UTC m=+35.163365794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:07.472823 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.472789    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-config-volume\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.477390 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.477236    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f37938d-32a7-444e-b325-ea4e94f96f31-cert\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.484173 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.484140    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krlf5\" (UniqueName: \"kubernetes.io/projected/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-kube-api-access-krlf5\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.484390 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.484372    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwvj6\" (UniqueName: \"kubernetes.io/projected/0f37938d-32a7-444e-b325-ea4e94f96f31-kube-api-access-fwvj6\") pod \"ingress-canary-nxzv5\" (UID: \"0f37938d-32a7-444e-b325-ea4e94f96f31\") " pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.529394 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.529360    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxzv5"
Jul 30 00:44:07.675515 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.675462    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxzv5"]
Jul 30 00:44:07.680558 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:07.680527    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f37938d_32a7_444e_b325_ea4e94f96f31.slice/crio-76fdcc84d4d10a74aa9de3c65c7367cfd4d0d546f544b1627f8c88c7d0c4f68f WatchSource:0}: Error finding container 76fdcc84d4d10a74aa9de3c65c7367cfd4d0d546f544b1627f8c88c7d0c4f68f: Status 404 returned error can't find the container with id 76fdcc84d4d10a74aa9de3c65c7367cfd4d0d546f544b1627f8c88c7d0c4f68f
Jul 30 00:44:07.684583 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.684317    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxzv5" event={"ID":"0f37938d-32a7-444e-b325-ea4e94f96f31","Type":"ContainerStarted","Data":"76fdcc84d4d10a74aa9de3c65c7367cfd4d0d546f544b1627f8c88c7d0c4f68f"}
Jul 30 00:44:07.977555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:07.977517    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:07.977710 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:07.977642    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:07.977710 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:07.977703    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:08.977689381 +0000 UTC m=+36.168637690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:08.988319 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:08.988203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:08.989305 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:08.988358    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:08.989305 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:08.988435    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:10.988415106 +0000 UTC m=+38.179363419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:10.696083 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:10.695786    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerStarted","Data":"658487e4d9639a866cef558e774e10b2ea6a009e8b3223c4a92b65fdf4155f29"}
Jul 30 00:44:11.003099 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:11.003068    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:11.003264 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:11.003208    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:11.003304 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:11.003266    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:15.003251855 +0000 UTC m=+42.194200187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:11.700633 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:11.700596    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="658487e4d9639a866cef558e774e10b2ea6a009e8b3223c4a92b65fdf4155f29" exitCode=0
Jul 30 00:44:11.701048 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:11.700681    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"658487e4d9639a866cef558e774e10b2ea6a009e8b3223c4a92b65fdf4155f29"}
Jul 30 00:44:11.702208 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:11.702189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxzv5" event={"ID":"0f37938d-32a7-444e-b325-ea4e94f96f31","Type":"ContainerStarted","Data":"581da7be0b7801170b8487b166deb7fea3160a0e5c031eaf2a2c0faced5c64b6"}
Jul 30 00:44:11.741139 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:11.741096    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nxzv5" podStartSLOduration=1.836078622 podStartE2EDuration="4.741080295s" totalImagesPullingTime="2.905001673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:11.740591107 +0000 UTC m=+38.931539485" watchObservedRunningTime="2026-07-30 00:44:11.741080295 +0000 UTC m=+38.932028630"
Jul 30 00:44:12.707645 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:12.707606    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3013612-b6cd-4e9f-9497-b4a528e08842" containerID="7e00a0b789800540f054809db5331fc4ecd3fb14b8edc00a1b810c77eb2dc95a" exitCode=0
Jul 30 00:44:12.708037 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:12.707696    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerDied","Data":"7e00a0b789800540f054809db5331fc4ecd3fb14b8edc00a1b810c77eb2dc95a"}
Jul 30 00:44:13.179064 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.179028    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"]
Jul 30 00:44:13.201473 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.201451    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"]
Jul 30 00:44:13.201605 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.201541    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"
Jul 30 00:44:13.204317 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.204292    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.205083 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.205064    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.205152 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.205108    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9pz7x\""
Jul 30 00:44:13.297251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.297216    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bb6d86d87-wsh2s"]
Jul 30 00:44:13.317919 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.317883    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"]
Jul 30 00:44:13.318087 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.317890    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.321220 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.321195    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 00:44:13.321475 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.321443    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt6vm\""
Jul 30 00:44:13.321579 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.321508    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 00:44:13.321752 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.321729    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 00:44:13.322707 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.322683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhx6c\" (UniqueName: \"kubernetes.io/projected/9971f216-379b-4067-95ca-34ce25bbea27-kube-api-access-rhx6c\") pod \"volume-data-source-validator-549c6d54cf-s6m82\" (UID: \"9971f216-379b-4067-95ca-34ce25bbea27\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"
Jul 30 00:44:13.330222 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.330198    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 00:44:13.335015 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.334997    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"]
Jul 30 00:44:13.335105 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.335032    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bb6d86d87-wsh2s"]
Jul 30 00:44:13.335162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.335132    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.337670 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.337649    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 30 00:44:13.337743 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.337704    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.338604 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.338591    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.338739 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.338713    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 30 00:44:13.338825 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.338719    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7pw4\""
Jul 30 00:44:13.377449 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.377417    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"]
Jul 30 00:44:13.399102 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.399077    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.402067 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402042    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.402386 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402370    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 30 00:44:13.402386 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402374    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.402539 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402471    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 30 00:44:13.402606 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b7jk5\""
Jul 30 00:44:13.402999 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.402979    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"]
Jul 30 00:44:13.423966 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.423933    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424124 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.423977    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424124 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424124 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424098    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424124 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424135    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwcwb\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424154    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.424251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhx6c\" (UniqueName: \"kubernetes.io/projected/9971f216-379b-4067-95ca-34ce25bbea27-kube-api-access-rhx6c\") pod \"volume-data-source-validator-549c6d54cf-s6m82\" (UID: \"9971f216-379b-4067-95ca-34ce25bbea27\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"
Jul 30 00:44:13.424251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.424203    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.433491 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.433428    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhx6c\" (UniqueName: \"kubernetes.io/projected/9971f216-379b-4067-95ca-34ce25bbea27-kube-api-access-rhx6c\") pod \"volume-data-source-validator-549c6d54cf-s6m82\" (UID: \"9971f216-379b-4067-95ca-34ce25bbea27\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"
Jul 30 00:44:13.510245 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.510210    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"
Jul 30 00:44:13.524574 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/156c239b-160e-4477-a71c-4ad6bca201af-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.524648 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.524648 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524601    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.524760 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.524842 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524826    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.524885 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524859    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.524924 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.524977 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525028 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.524982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwcwb\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525234 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx4ts\" (UniqueName: \"kubernetes.io/projected/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-kube-api-access-kx4ts\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.525234 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525235    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnvdl\" (UniqueName: \"kubernetes.io/projected/156c239b-160e-4477-a71c-4ad6bca201af-kube-api-access-bnvdl\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.525409 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525409 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525352    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525409 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525360    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.525550 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:13.525410    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:13.525550 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:13.525423    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:13.525550 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:13.525480    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.025465984 +0000 UTC m=+41.216414313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:13.525662 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.525565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.526978 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.526955    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.527171 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.527153    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.527579 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.527560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.533455 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.533435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.533635 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.533587    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwcwb\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:13.625891 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.625849    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx4ts\" (UniqueName: \"kubernetes.io/projected/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-kube-api-access-kx4ts\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.625891 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.625887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnvdl\" (UniqueName: \"kubernetes.io/projected/156c239b-160e-4477-a71c-4ad6bca201af-kube-api-access-bnvdl\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.626115 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.625945    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/156c239b-160e-4477-a71c-4ad6bca201af-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.626115 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.625964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.626115 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.626108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.626282 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.626152    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.626282 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:13.626220    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:13.626436 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:13.626305    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.126262713 +0000 UTC m=+41.317211027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:13.626666 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.626646    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.626730 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.626697    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/156c239b-160e-4477-a71c-4ad6bca201af-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.628068 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.628045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.634244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.634216    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnvdl\" (UniqueName: \"kubernetes.io/projected/156c239b-160e-4477-a71c-4ad6bca201af-kube-api-access-bnvdl\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:13.634606 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.634587    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx4ts\" (UniqueName: \"kubernetes.io/projected/cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de-kube-api-access-kx4ts\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp\" (UID: \"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.681459 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.681425    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82"]
Jul 30 00:44:13.684589 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:13.684517    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9971f216_379b_4067_95ca_34ce25bbea27.slice/crio-4904f376200bc9e7256ba029589f4057bbd539e5ecf01a0c620f329404b66d2e WatchSource:0}: Error finding container 4904f376200bc9e7256ba029589f4057bbd539e5ecf01a0c620f329404b66d2e: Status 404 returned error can't find the container with id 4904f376200bc9e7256ba029589f4057bbd539e5ecf01a0c620f329404b66d2e
Jul 30 00:44:13.708260 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.708232    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"
Jul 30 00:44:13.713246 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.713220    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" event={"ID":"e3013612-b6cd-4e9f-9497-b4a528e08842","Type":"ContainerStarted","Data":"1daad85c750ac4c3138e8f9f0f43f872a3df1f81ed794129e60862c00691ef59"}
Jul 30 00:44:13.714269 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.714248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82" event={"ID":"9971f216-379b-4067-95ca-34ce25bbea27","Type":"ContainerStarted","Data":"4904f376200bc9e7256ba029589f4057bbd539e5ecf01a0c620f329404b66d2e"}
Jul 30 00:44:13.734378 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.734340    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xnfrk" podStartSLOduration=7.168360375 podStartE2EDuration="40.734325769s" totalImagesPullingTime="33.565965394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:13.732520827 +0000 UTC m=+40.923469162" watchObservedRunningTime="2026-07-30 00:44:13.734325769 +0000 UTC m=+40.925274100"
Jul 30 00:44:13.847374 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:13.847352    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp"]
Jul 30 00:44:13.849837 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:13.849790    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbf34d4_88f0_4f0c_ba96_b5f5ef8147de.slice/crio-29cf44fd10c7f763ff85a683b66a307f0403ce83f9e724d1fd63ecfe7d0681a4 WatchSource:0}: Error finding container 29cf44fd10c7f763ff85a683b66a307f0403ce83f9e724d1fd63ecfe7d0681a4: Status 404 returned error can't find the container with id 29cf44fd10c7f763ff85a683b66a307f0403ce83f9e724d1fd63ecfe7d0681a4
Jul 30 00:44:14.030177 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:14.030147    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:14.030349 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:14.030253    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:14.030349 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:14.030265    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:14.030349 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:14.030324    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:15.030308173 +0000 UTC m=+42.221256481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:14.131087 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:14.131050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:14.131238 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:14.131200    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:14.131282 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:14.131260    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:15.131245909 +0000 UTC m=+42.322194217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:14.718478 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:14.718443    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp" event={"ID":"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de","Type":"ContainerStarted","Data":"29cf44fd10c7f763ff85a683b66a307f0403ce83f9e724d1fd63ecfe7d0681a4"}
Jul 30 00:44:15.037611 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:15.037571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:15.037786 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:15.037636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:15.037786 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.037727    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:15.037786 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.037751    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:15.037786 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.037726    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:15.037986 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.037830    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:17.037814202 +0000 UTC m=+44.228762511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:15.037986 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.037843    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.037837196 +0000 UTC m=+50.228785505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:15.138400 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:15.138358    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:15.138568 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.138521    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:15.138618 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:15.138591    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:17.138575361 +0000 UTC m=+44.329523670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:15.723543 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:15.723508    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82" event={"ID":"9971f216-379b-4067-95ca-34ce25bbea27","Type":"ContainerStarted","Data":"1b1c0dad4f20fc83cdc6771b43a18754d1d5ad3047d7dafdea1926fa32d04168"}
Jul 30 00:44:16.741898 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:16.741839    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-s6m82" podStartSLOduration=1.792130427 podStartE2EDuration="3.74181929s" totalImagesPullingTime="1.949688863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:16.740766599 +0000 UTC m=+43.931714931" watchObservedRunningTime="2026-07-30 00:44:16.74181929 +0000 UTC m=+43.932767617"
Jul 30 00:44:17.051574 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:17.051481    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:17.051738 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:17.051670    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:17.051738 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:17.051697    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:17.051896 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:17.051784    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.051761618 +0000 UTC m=+48.242709950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:17.152113 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:17.152075    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:17.152278 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:17.152223    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:17.152330 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:17.152292    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.152275568 +0000 UTC m=+48.343223877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:17.730781 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:17.730746    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp" event={"ID":"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de","Type":"ContainerStarted","Data":"26e5af4c24cace43d1742eabf7ccd109dd218fde1d5a5c5089fce3935884fddb"}
Jul 30 00:44:17.745611 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:17.745565    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp" podStartSLOduration=1.084319556 podStartE2EDuration="4.745551545s" totalImagesPullingTime="3.661231989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:17.744527303 +0000 UTC m=+44.935475635" watchObservedRunningTime="2026-07-30 00:44:17.745551545 +0000 UTC m=+44.936499919"
Jul 30 00:44:21.086741 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.086700    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:21.087143 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.086876    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:21.087143 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.086896    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:21.087143 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.086959    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.08694165 +0000 UTC m=+56.277889982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:21.187764 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.187732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:21.187921 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.187900    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:21.187976 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.187969    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.187953177 +0000 UTC m=+56.378901511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:21.322241 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.322212    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tjr6x"]
Jul 30 00:44:21.362261 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.362173    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tjr6x"]
Jul 30 00:44:21.362430 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.362302    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.364441 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.364414    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 00:44:21.364736 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.364713    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 00:44:21.364830 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.364741    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 00:44:21.365548 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.365528    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 00:44:21.365631 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.365535    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p9rd8\""
Jul 30 00:44:21.389012 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.388989    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wqq\" (UniqueName: \"kubernetes.io/projected/fe11ab37-4387-4566-ae2c-3febc2522285-kube-api-access-x5wqq\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.389105 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.389023    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe11ab37-4387-4566-ae2c-3febc2522285-crio-socket\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.389105 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.389042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe11ab37-4387-4566-ae2c-3febc2522285-data-volume\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.389180 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.389140    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.389180 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.389169    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489617 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489759 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wqq\" (UniqueName: \"kubernetes.io/projected/fe11ab37-4387-4566-ae2c-3febc2522285-kube-api-access-x5wqq\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489759 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe11ab37-4387-4566-ae2c-3febc2522285-crio-socket\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489759 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.489738    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:21.489759 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489744    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe11ab37-4387-4566-ae2c-3febc2522285-data-volume\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489972 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489843    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.489972 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.489893    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.989863468 +0000 UTC m=+49.180811780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:21.489972 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.489919    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.490096 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.489971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe11ab37-4387-4566-ae2c-3febc2522285-crio-socket\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.490096 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.489988    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.989971989 +0000 UTC m=+49.180920315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.490183 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.490166    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe11ab37-4387-4566-ae2c-3febc2522285-data-volume\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.500058 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.500038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wqq\" (UniqueName: \"kubernetes.io/projected/fe11ab37-4387-4566-ae2c-3febc2522285-kube-api-access-x5wqq\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.993867 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.993823    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.993867 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:21.993869    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:21.994102 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.993936    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.994102 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.993961    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:21.994102 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.994013    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.993998213 +0000 UTC m=+50.184946522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.994102 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:21.994028    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.994022335 +0000 UTC m=+50.184970644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:23.000884 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.000848    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:23.000884 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.000886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:23.001276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.000970    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:23.001276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.000998    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:23.001276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.001036    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.001018025 +0000 UTC m=+52.191966352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:23.001276 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.001051    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.001044022 +0000 UTC m=+52.191992330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:23.101915 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.101875    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:23.102090 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.102024    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:23.102090 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:23.102083    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls podName:dc4b2be7-6a3d-4f54-acfc-37a1d42b5754 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:39.102068865 +0000 UTC m=+66.293017199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls") pod "dns-default-t69pf" (UID: "dc4b2be7-6a3d-4f54-acfc-37a1d42b5754") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:23.363483 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.363387    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-s4g96"]
Jul 30 00:44:23.382667 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.382634    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-s4g96"]
Jul 30 00:44:23.382855 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.382753    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.385185 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.385162    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 30 00:44:23.385312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.385251    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 30 00:44:23.387384 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.387364    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 30 00:44:23.387540 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.387518    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-k6r7f\""
Jul 30 00:44:23.387675 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.387615    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 30 00:44:23.404966 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.404944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-cabundle\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.405061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.405042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq769\" (UniqueName: \"kubernetes.io/projected/d83492dd-31cb-4b1e-a3b4-b01595785690-kube-api-access-vq769\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.405129 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.405117    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-key\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.505658 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.505618    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-cabundle\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.505855 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.505696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq769\" (UniqueName: \"kubernetes.io/projected/d83492dd-31cb-4b1e-a3b4-b01595785690-kube-api-access-vq769\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.505855 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.505737    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-key\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.506287 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.506262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-cabundle\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.508151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.508126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d83492dd-31cb-4b1e-a3b4-b01595785690-signing-key\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.513300 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.513278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq769\" (UniqueName: \"kubernetes.io/projected/d83492dd-31cb-4b1e-a3b4-b01595785690-kube-api-access-vq769\") pod \"service-ca-658d7b7c78-s4g96\" (UID: \"d83492dd-31cb-4b1e-a3b4-b01595785690\") " pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.694116 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.694036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-s4g96"
Jul 30 00:44:23.817060 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.817026    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-s4g96"]
Jul 30 00:44:23.820395 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:23.820365    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83492dd_31cb_4b1e_a3b4_b01595785690.slice/crio-3a9492444ecbe84dfe2f18e46ea589e216f26a4eec050018c1e738ba446c52e3 WatchSource:0}: Error finding container 3a9492444ecbe84dfe2f18e46ea589e216f26a4eec050018c1e738ba446c52e3: Status 404 returned error can't find the container with id 3a9492444ecbe84dfe2f18e46ea589e216f26a4eec050018c1e738ba446c52e3
Jul 30 00:44:23.904109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:23.904084    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-62tnh_86d15d74-beff-4568-9c89-660ad1fca368/dns-node-resolver/0.log"
Jul 30 00:44:24.751444 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:24.751408    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-s4g96" event={"ID":"d83492dd-31cb-4b1e-a3b4-b01595785690","Type":"ContainerStarted","Data":"3a9492444ecbe84dfe2f18e46ea589e216f26a4eec050018c1e738ba446c52e3"}
Jul 30 00:44:25.020671 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:25.020593    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:25.020858 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:25.020672    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:25.020858 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:25.020828    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:25.020980 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:25.020856    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:25.020980 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:25.020908    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.020885353 +0000 UTC m=+56.211833685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:25.020980 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:25.020931    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.020920602 +0000 UTC m=+56.211868918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:25.305051 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:25.304972    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xv4mc_1301c9a9-7463-4730-9548-8fc0a02803cb/node-ca/0.log"
Jul 30 00:44:26.104844 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:26.104819    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxzv5_0f37938d-32a7-444e-b325-ea4e94f96f31/serve-healthcheck-canary/0.log"
Jul 30 00:44:26.705511 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:26.705481    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp_cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de/kube-storage-version-migrator-operator/0.log"
Jul 30 00:44:26.757426 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:26.757399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-s4g96" event={"ID":"d83492dd-31cb-4b1e-a3b4-b01595785690","Type":"ContainerStarted","Data":"ab0254543ec2cb7b7f1dd925a2d348ab09795f9dc179f2fc40bcb55bdcab18eb"}
Jul 30 00:44:26.773360 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:26.773308    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-s4g96" podStartSLOduration=1.514577514 podStartE2EDuration="3.773293241s" totalImagesPullingTime="2.258715727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:26.77228026 +0000 UTC m=+53.963228595" watchObservedRunningTime="2026-07-30 00:44:26.773293241 +0000 UTC m=+53.964241587"
Jul 30 00:44:29.053425 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:29.053372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:29.053809 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:29.053470    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:29.053809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.053556    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:29.053809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.053582    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:29.053809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.053638    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.053622401 +0000 UTC m=+64.244570710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:29.053809 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.053655    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.053648656 +0000 UTC m=+64.244596965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:29.154866 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:29.154822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:29.155052 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.154955    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 00:44:29.155052 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.154971    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bb6d86d87-wsh2s: secret "image-registry-tls" not found
Jul 30 00:44:29.155052 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.155019    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls podName:eb2ec8b1-2c3b-40b5-bef7-28820939e7f9 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:45.155004253 +0000 UTC m=+72.345952563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls") pod "image-registry-5bb6d86d87-wsh2s" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9") : secret "image-registry-tls" not found
Jul 30 00:44:29.255354 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:29.255317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:29.255544 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.255435    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:29.255544 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:29.255484    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls podName:156c239b-160e-4477-a71c-4ad6bca201af nodeName:}" failed. No retries permitted until 2026-07-30 00:44:45.255471574 +0000 UTC m=+72.446419884 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-hvnbr" (UID: "156c239b-160e-4477-a71c-4ad6bca201af") : secret "cluster-monitoring-operator-tls" not found
Jul 30 00:44:31.674516 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:31.674485    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vdlqw"
Jul 30 00:44:37.120057 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:37.120010    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:37.120502 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:37.120101    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:37.120502 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:37.120153    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:37.120502 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:37.120220    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls podName:fe11ab37-4387-4566-ae2c-3febc2522285 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:53.120204657 +0000 UTC m=+80.311152967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tjr6x" (UID: "fe11ab37-4387-4566-ae2c-3febc2522285") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:37.120613 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:37.120598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe11ab37-4387-4566-ae2c-3febc2522285-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:38.128894 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.128851    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:38.128894 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.128895    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:38.132220 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.132199    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 00:44:38.133247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.133230    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 00:44:38.141509 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.141474    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b10cef9e-bbe3-45a5-aefd-30c11fb84b4d-original-pull-secret\") pod \"global-pull-secret-syncer-j69sg\" (UID: \"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d\") " pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:38.141644 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.141569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3fb6af-f27f-4c36-ac55-d12613327f01-metrics-certs\") pod \"network-metrics-daemon-qb96l\" (UID: \"9b3fb6af-f27f-4c36-ac55-d12613327f01\") " pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:38.230234 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.230197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:38.233092 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.233074    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 00:44:38.242983 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.242968    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 00:44:38.254392 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.254363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqs5q\" (UniqueName: \"kubernetes.io/projected/cd8796e7-7571-4ead-8b4e-bc6dbe502ca4-kube-api-access-lqs5q\") pod \"network-check-target-gxgks\" (UID: \"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4\") " pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:38.291389 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.291354    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j69sg"
Jul 30 00:44:38.300661 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.300567    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\""
Jul 30 00:44:38.306471 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.306450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\""
Jul 30 00:44:38.309269 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.309250    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:38.315304 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.315275    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb96l"
Jul 30 00:44:38.433907 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.433873    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j69sg"]
Jul 30 00:44:38.437624 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:38.437596    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10cef9e_bbe3_45a5_aefd_30c11fb84b4d.slice/crio-8518bcf1c884a9afa021c94349cf796deb876ca5b5b3ac3c4fed8581807d4252 WatchSource:0}: Error finding container 8518bcf1c884a9afa021c94349cf796deb876ca5b5b3ac3c4fed8581807d4252: Status 404 returned error can't find the container with id 8518bcf1c884a9afa021c94349cf796deb876ca5b5b3ac3c4fed8581807d4252
Jul 30 00:44:38.451261 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.451236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxgks"]
Jul 30 00:44:38.454121 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:38.454095    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8796e7_7571_4ead_8b4e_bc6dbe502ca4.slice/crio-66beae3667a50d016f9cfa049c370f8d1464561df2a6d7c319cd30757c71dff6 WatchSource:0}: Error finding container 66beae3667a50d016f9cfa049c370f8d1464561df2a6d7c319cd30757c71dff6: Status 404 returned error can't find the container with id 66beae3667a50d016f9cfa049c370f8d1464561df2a6d7c319cd30757c71dff6
Jul 30 00:44:38.463422 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.463394    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qb96l"]
Jul 30 00:44:38.466770 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:38.466744    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3fb6af_f27f_4c36_ac55_d12613327f01.slice/crio-78b2ae698d07b97350ea09743c06363381da2685e1b6ba5e5a810e459842fd17 WatchSource:0}: Error finding container 78b2ae698d07b97350ea09743c06363381da2685e1b6ba5e5a810e459842fd17: Status 404 returned error can't find the container with id 78b2ae698d07b97350ea09743c06363381da2685e1b6ba5e5a810e459842fd17
Jul 30 00:44:38.789195 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.789160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb96l" event={"ID":"9b3fb6af-f27f-4c36-ac55-d12613327f01","Type":"ContainerStarted","Data":"78b2ae698d07b97350ea09743c06363381da2685e1b6ba5e5a810e459842fd17"}
Jul 30 00:44:38.790258 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.790233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxgks" event={"ID":"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4","Type":"ContainerStarted","Data":"66beae3667a50d016f9cfa049c370f8d1464561df2a6d7c319cd30757c71dff6"}
Jul 30 00:44:38.794880 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:38.794815    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j69sg" event={"ID":"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d","Type":"ContainerStarted","Data":"8518bcf1c884a9afa021c94349cf796deb876ca5b5b3ac3c4fed8581807d4252"}
Jul 30 00:44:39.138129 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:39.138037    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:39.141432 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:39.141371    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc4b2be7-6a3d-4f54-acfc-37a1d42b5754-metrics-tls\") pod \"dns-default-t69pf\" (UID: \"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754\") " pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:39.325608 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:39.325575    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\""
Jul 30 00:44:39.333590 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:39.333555    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:39.517777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:39.517742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t69pf"]
Jul 30 00:44:39.867516 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:39.867475    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4b2be7_6a3d_4f54_acfc_37a1d42b5754.slice/crio-eaf00f75a3fb25c62197bf228546d2e2b4c991882a992514ce1bf62a772b94bb WatchSource:0}: Error finding container eaf00f75a3fb25c62197bf228546d2e2b4c991882a992514ce1bf62a772b94bb: Status 404 returned error can't find the container with id eaf00f75a3fb25c62197bf228546d2e2b4c991882a992514ce1bf62a772b94bb
Jul 30 00:44:40.806821 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:40.806768    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb96l" event={"ID":"9b3fb6af-f27f-4c36-ac55-d12613327f01","Type":"ContainerStarted","Data":"3e9f0f1357b15a11ebb6986d34b92ef9d4fce3ce34ee2ad51ea7c276e2d15719"}
Jul 30 00:44:40.807275 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:40.806853    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb96l" event={"ID":"9b3fb6af-f27f-4c36-ac55-d12613327f01","Type":"ContainerStarted","Data":"4bfab991a161ed74c7f7eb0e189fd59bae3284426d1b75541a905db89a7d86cf"}
Jul 30 00:44:40.808618 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:40.808585    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69pf" event={"ID":"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754","Type":"ContainerStarted","Data":"eaf00f75a3fb25c62197bf228546d2e2b4c991882a992514ce1bf62a772b94bb"}
Jul 30 00:44:40.823807 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:40.823744    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qb96l" podStartSLOduration=66.100847101 podStartE2EDuration="1m7.823724034s" totalImagesPullingTime="1.722876933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:40.822069175 +0000 UTC m=+68.013017508" watchObservedRunningTime="2026-07-30 00:44:40.823724034 +0000 UTC m=+68.014672368"
Jul 30 00:44:43.822436 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.822402    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxgks" event={"ID":"cd8796e7-7571-4ead-8b4e-bc6dbe502ca4","Type":"ContainerStarted","Data":"49fefdb6ce3a5edd83b270421f8168570939be45b6c69b63697988316ef0eeea"}
Jul 30 00:44:43.822891 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.822504    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:44:43.824288 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.824267    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69pf" event={"ID":"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754","Type":"ContainerStarted","Data":"a48a128b70d6e3c005f752cf6173d76159076271fb218fe3c5e3bea10fead4a3"}
Jul 30 00:44:43.825586 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.825563    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j69sg" event={"ID":"b10cef9e-bbe3-45a5-aefd-30c11fb84b4d","Type":"ContainerStarted","Data":"b9a29c53b677118e0cd032eb937f54a4dc6c9629318267447c7764b206b3cdb9"}
Jul 30 00:44:43.836768 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.836526    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gxgks" podStartSLOduration=65.641523357 podStartE2EDuration="1m10.836510721s" totalImagesPullingTime="5.194987364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:43.836131573 +0000 UTC m=+71.027079905" watchObservedRunningTime="2026-07-30 00:44:43.836510721 +0000 UTC m=+71.027459052"
Jul 30 00:44:43.849888 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:43.849849    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j69sg" podStartSLOduration=65.627173639 podStartE2EDuration="1m10.849835445s" totalImagesPullingTime="5.222661806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:43.849243036 +0000 UTC m=+71.040191364" watchObservedRunningTime="2026-07-30 00:44:43.849835445 +0000 UTC m=+71.040783776"
Jul 30 00:44:44.123061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.123028    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"]
Jul 30 00:44:44.124905 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.124888    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.127194 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.127167    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c5wwk\""
Jul 30 00:44:44.128177 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.128162    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 00:44:44.128247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.128227    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 00:44:44.128305 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.128253    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 00:44:44.128305 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.128224    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 00:44:44.136899 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.136878    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"]
Jul 30 00:44:44.145556 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.145529    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bb6d86d87-wsh2s"]
Jul 30 00:44:44.145721 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:44.145705    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s" podUID="eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"
Jul 30 00:44:44.181505 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.181476    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.181648 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.181510    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48kmm\" (UniqueName: \"kubernetes.io/projected/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-kube-api-access-48kmm\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.182808 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.182773    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"]
Jul 30 00:44:44.184588 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.184572    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.201024 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.200996    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"]
Jul 30 00:44:44.282719 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282694    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-trusted-ca\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282883 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-installation-pull-secrets\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282883 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282830    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-tls\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282891    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-ca-trust-extracted\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282933    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-certificates\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9wp\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-kube-api-access-4r9wp\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.282990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.283164 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.282999    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kmm\" (UniqueName: \"kubernetes.io/projected/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-kube-api-access-48kmm\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.283164 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.283046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-image-registry-private-configuration\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.283164 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.283080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-bound-sa-token\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.285426 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.285408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.293502 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.293484    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48kmm\" (UniqueName: \"kubernetes.io/projected/98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672-kube-api-access-48kmm\") pod \"managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp\" (UID: \"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.384327 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-image-registry-private-configuration\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384327 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-bound-sa-token\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384327 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-trusted-ca\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384327 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-installation-pull-secrets\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384655 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-tls\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384655 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384399    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-ca-trust-extracted\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384655 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384431    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-certificates\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384655 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9wp\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-kube-api-access-4r9wp\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.384973 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.384935    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-ca-trust-extracted\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.385540 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.385516    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-certificates\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.386308 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.386281    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-trusted-ca\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.387030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.387009    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-image-registry-private-configuration\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.387169 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.387153    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-installation-pull-secrets\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.387791 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.387773    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-registry-tls\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.395221 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.395203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-bound-sa-token\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.395460 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.395444    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9wp\" (UniqueName: \"kubernetes.io/projected/91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd-kube-api-access-4r9wp\") pod \"image-registry-7fc6ddfd74-lbz2m\" (UID: \"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd\") " pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.445924 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.445897    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"
Jul 30 00:44:44.495144 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.495104    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt6vm\""
Jul 30 00:44:44.503244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.503215    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.575925 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.575900    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp"]
Jul 30 00:44:44.578733 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:44.578701    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98e12a65_b6e5_4d2e_a7e5_6fb93f1bf672.slice/crio-d4de3d89841998089debeafa379c737f37b38ade7a09e2b80a09167ae611f17b WatchSource:0}: Error finding container d4de3d89841998089debeafa379c737f37b38ade7a09e2b80a09167ae611f17b: Status 404 returned error can't find the container with id d4de3d89841998089debeafa379c737f37b38ade7a09e2b80a09167ae611f17b
Jul 30 00:44:44.635278 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.635175    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"]
Jul 30 00:44:44.637736 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:44.637704    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cb2a4f_8c53_4ae4_be1d_07eac2de3dcd.slice/crio-5d184dac32e070158caf81cbef79bf125691c6d1bfb980e4a6742203acc02eb2 WatchSource:0}: Error finding container 5d184dac32e070158caf81cbef79bf125691c6d1bfb980e4a6742203acc02eb2: Status 404 returned error can't find the container with id 5d184dac32e070158caf81cbef79bf125691c6d1bfb980e4a6742203acc02eb2
Jul 30 00:44:44.830903 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.830864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69pf" event={"ID":"dc4b2be7-6a3d-4f54-acfc-37a1d42b5754","Type":"ContainerStarted","Data":"277d27a2d48f4622bfcb4da7416e2e89da6ed01443257b24ca5b9400f473585b"}
Jul 30 00:44:44.831322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.831001    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:44.832070 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.832044    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp" event={"ID":"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672","Type":"ContainerStarted","Data":"d4de3d89841998089debeafa379c737f37b38ade7a09e2b80a09167ae611f17b"}
Jul 30 00:44:44.833611 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.833532    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:44.833611 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.833529    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m" event={"ID":"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd","Type":"ContainerStarted","Data":"aece5ff84b37b394eec3558d39f2c4e32775b76837b37884be63f9086dba1371"}
Jul 30 00:44:44.833611 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.833597    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m" event={"ID":"91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd","Type":"ContainerStarted","Data":"5d184dac32e070158caf81cbef79bf125691c6d1bfb980e4a6742203acc02eb2"}
Jul 30 00:44:44.833863 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.833658    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:44:44.838689 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.838672    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:44.847231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.847196    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t69pf" podStartSLOduration=34.061235043 podStartE2EDuration="37.847183645s" totalImagesPullingTime="3.785948602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:44.846690222 +0000 UTC m=+72.037638553" watchObservedRunningTime="2026-07-30 00:44:44.847183645 +0000 UTC m=+72.038131975"
Jul 30 00:44:44.864982 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.864922    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m" podStartSLOduration=0.864902343 podStartE2EDuration="864.902343ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:44.863537956 +0000 UTC m=+72.054486298" watchObservedRunningTime="2026-07-30 00:44:44.864902343 +0000 UTC m=+72.055850674"
Jul 30 00:44:44.889083 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889015    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889083 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889053    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889083 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889077    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889143    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889205    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwcwb\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889243    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889268    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:44.889323 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889308    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:44:44.889548 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889494    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:44:44.889831 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889759    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-trusted-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.889831 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.889788    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-ca-trust-extracted\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.890364 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.890335    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:44:44.891867 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.891839    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:44:44.891997 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.891960    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb" (OuterVolumeSpecName: "kube-api-access-cwcwb") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "kube-api-access-cwcwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:44:44.892414 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.892392    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:44:44.892548 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.892525    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:44:44.991183 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.991147    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-installation-pull-secrets\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.991183 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.991175    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwcwb\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-kube-api-access-cwcwb\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.991183 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.991185    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-image-registry-private-configuration\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.991405 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.991195    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-certificates\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:44.991405 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:44.991208    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-bound-sa-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:45.193022 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.192931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:45.195256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.195225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"image-registry-5bb6d86d87-wsh2s\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") " pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:45.294276 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.294246    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") pod \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\" (UID: \"eb2ec8b1-2c3b-40b5-bef7-28820939e7f9\") "
Jul 30 00:44:45.294467 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.294442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:45.296468 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.296423    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" (UID: "eb2ec8b1-2c3b-40b5-bef7-28820939e7f9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:44:45.297248 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.297223    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/156c239b-160e-4477-a71c-4ad6bca201af-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-hvnbr\" (UID: \"156c239b-160e-4477-a71c-4ad6bca201af\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:45.395807 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.395763    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9-registry-tls\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:44:45.447422 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.447340    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7pw4\""
Jul 30 00:44:45.454472 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.454437    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"
Jul 30 00:44:45.606447 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.606367    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr"]
Jul 30 00:44:45.608708 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:45.608681    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156c239b_160e_4477_a71c_4ad6bca201af.slice/crio-ebf5992acc40d9ae4b691bb62ca4ed761a0d1280914379d90b1a1f0074818cae WatchSource:0}: Error finding container ebf5992acc40d9ae4b691bb62ca4ed761a0d1280914379d90b1a1f0074818cae: Status 404 returned error can't find the container with id ebf5992acc40d9ae4b691bb62ca4ed761a0d1280914379d90b1a1f0074818cae
Jul 30 00:44:45.841036 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.840996    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr" event={"ID":"156c239b-160e-4477-a71c-4ad6bca201af","Type":"ContainerStarted","Data":"ebf5992acc40d9ae4b691bb62ca4ed761a0d1280914379d90b1a1f0074818cae"}
Jul 30 00:44:45.841485 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.841295    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bb6d86d87-wsh2s"
Jul 30 00:44:45.869852 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.869821    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bb6d86d87-wsh2s"]
Jul 30 00:44:45.873442 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:45.873412    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bb6d86d87-wsh2s"]
Jul 30 00:44:47.384412 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:47.384373    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb2ec8b1-2c3b-40b5-bef7-28820939e7f9" path="/var/lib/kubelet/pods/eb2ec8b1-2c3b-40b5-bef7-28820939e7f9/volumes"
Jul 30 00:44:47.850192 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:47.850151    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp" event={"ID":"98e12a65-b6e5-4d2e-a7e5-6fb93f1bf672","Type":"ContainerStarted","Data":"1f97764f0518735fe1f3ed97132d22c2128b68f63bd42f74fa4b9b8366128047"}
Jul 30 00:44:47.868118 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:47.868062    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cfb8bcc6-4z8kp" podStartSLOduration=1.481524382 podStartE2EDuration="3.868044208s" totalImagesPullingTime="2.386519826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:47.866863224 +0000 UTC m=+75.057811557" watchObservedRunningTime="2026-07-30 00:44:47.868044208 +0000 UTC m=+75.058992539"
Jul 30 00:44:48.753130 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.753101    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"]
Jul 30 00:44:48.756162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.756145    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:48.758404 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.758373    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 30 00:44:48.758523 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.758445    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-t7c68\""
Jul 30 00:44:48.763651 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.763630    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"]
Jul 30 00:44:48.826986 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.826951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-l4k7m\" (UID: \"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:48.855742 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.855710    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr" event={"ID":"156c239b-160e-4477-a71c-4ad6bca201af","Type":"ContainerStarted","Data":"a513fbf77f8ea552cf71cd638e478fe1c7fe79a178b44387bea16e7645b8b1cf"}
Jul 30 00:44:48.928115 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:48.928037    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-l4k7m\" (UID: \"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:48.928265 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:48.928176    2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 30 00:44:48.928265 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:48.928230    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates podName:fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:49.428216379 +0000 UTC m=+76.619164690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-l4k7m" (UID: "fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 30 00:44:49.433360 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.433329    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-l4k7m\" (UID: \"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:49.435718 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.435700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-l4k7m\" (UID: \"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:49.665936 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.665902    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:49.811222 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.811161    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-hvnbr" podStartSLOduration=34.132328739 podStartE2EDuration="36.81113705s" totalImagesPullingTime="2.678808311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:48.87147814 +0000 UTC m=+76.062426471" watchObservedRunningTime="2026-07-30 00:44:49.81113705 +0000 UTC m=+77.002085383"
Jul 30 00:44:49.812111 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.812089    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"]
Jul 30 00:44:49.814749 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:49.814727    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0a4fb4_4ed1_4dd9_85d2_7f0e12416856.slice/crio-63dbc748363dcb0925c641982821ca4c72b705621d23666e2f0c2a282475bc1e WatchSource:0}: Error finding container 63dbc748363dcb0925c641982821ca4c72b705621d23666e2f0c2a282475bc1e: Status 404 returned error can't find the container with id 63dbc748363dcb0925c641982821ca4c72b705621d23666e2f0c2a282475bc1e
Jul 30 00:44:49.859843 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:49.859815    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m" event={"ID":"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856","Type":"ContainerStarted","Data":"63dbc748363dcb0925c641982821ca4c72b705621d23666e2f0c2a282475bc1e"}
Jul 30 00:44:51.870754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:51.870711    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m" event={"ID":"fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856","Type":"ContainerStarted","Data":"7373cf593870ac04f173746fc89666edf892842c26528c945c0402ac723da6db"}
Jul 30 00:44:51.871153 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:51.870933    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:51.875311 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:51.875289    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m"
Jul 30 00:44:51.887049 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:51.887007    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-l4k7m" podStartSLOduration=2.850080327 podStartE2EDuration="3.88699607s" totalImagesPullingTime="1.036915743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:51.885671324 +0000 UTC m=+79.076619655" watchObservedRunningTime="2026-07-30 00:44:51.88699607 +0000 UTC m=+79.077944400"
Jul 30 00:44:52.808098 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.808062    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h4z85"]
Jul 30 00:44:52.811242 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.811221    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:52.814939 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.814917    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 30 00:44:52.815080 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.814979    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 30 00:44:52.815157 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.815084    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rqm5r\""
Jul 30 00:44:52.815225 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.815190    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 00:44:52.821394 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.821368    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h4z85"]
Jul 30 00:44:52.961201 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.961167    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp4hz\" (UniqueName: \"kubernetes.io/projected/f08ef237-f02a-42b4-b935-6abb62d0eaf9-kube-api-access-rp4hz\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:52.961609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.961272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f08ef237-f02a-42b4-b935-6abb62d0eaf9-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:52.961609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.961326    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:52.961609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:52.961354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.062109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.061996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.062109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.062059    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.062109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.062114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp4hz\" (UniqueName: \"kubernetes.io/projected/f08ef237-f02a-42b4-b935-6abb62d0eaf9-kube-api-access-rp4hz\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.062380 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.062170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f08ef237-f02a-42b4-b935-6abb62d0eaf9-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.062789 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.062764    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f08ef237-f02a-42b4-b935-6abb62d0eaf9-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.064518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.064487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.064607 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.064498    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f08ef237-f02a-42b4-b935-6abb62d0eaf9-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.070416 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.070389    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp4hz\" (UniqueName: \"kubernetes.io/projected/f08ef237-f02a-42b4-b935-6abb62d0eaf9-kube-api-access-rp4hz\") pod \"prometheus-operator-84d69f794d-h4z85\" (UID: \"f08ef237-f02a-42b4-b935-6abb62d0eaf9\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.119905 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.119880    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85"
Jul 30 00:44:53.163452 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.163415    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:53.167045 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.166725    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe11ab37-4387-4566-ae2c-3febc2522285-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tjr6x\" (UID: \"fe11ab37-4387-4566-ae2c-3febc2522285\") " pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:53.173902 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.173877    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p9rd8\""
Jul 30 00:44:53.181955 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.181932    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tjr6x"
Jul 30 00:44:53.247524 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.247498    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h4z85"]
Jul 30 00:44:53.250527 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:53.250497    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08ef237_f02a_42b4_b935_6abb62d0eaf9.slice/crio-1766b1a15f5df55884db3c54be28e3cdc17e420446d0e94420e2abe5fc97f90b WatchSource:0}: Error finding container 1766b1a15f5df55884db3c54be28e3cdc17e420446d0e94420e2abe5fc97f90b: Status 404 returned error can't find the container with id 1766b1a15f5df55884db3c54be28e3cdc17e420446d0e94420e2abe5fc97f90b
Jul 30 00:44:53.314161 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.314072    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tjr6x"]
Jul 30 00:44:53.318001 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:53.317975    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe11ab37_4387_4566_ae2c_3febc2522285.slice/crio-1cfe55e9037387bdd6938e732ac3f46e8b2a0ecc52d21f08f5cd033d811e891b WatchSource:0}: Error finding container 1cfe55e9037387bdd6938e732ac3f46e8b2a0ecc52d21f08f5cd033d811e891b: Status 404 returned error can't find the container with id 1cfe55e9037387bdd6938e732ac3f46e8b2a0ecc52d21f08f5cd033d811e891b
Jul 30 00:44:53.881420 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.881387    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tjr6x" event={"ID":"fe11ab37-4387-4566-ae2c-3febc2522285","Type":"ContainerStarted","Data":"1cfe55e9037387bdd6938e732ac3f46e8b2a0ecc52d21f08f5cd033d811e891b"}
Jul 30 00:44:53.882500 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:53.882471    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85" event={"ID":"f08ef237-f02a-42b4-b935-6abb62d0eaf9","Type":"ContainerStarted","Data":"1766b1a15f5df55884db3c54be28e3cdc17e420446d0e94420e2abe5fc97f90b"}
Jul 30 00:44:54.842925 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:54.842819    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t69pf"
Jul 30 00:44:55.892500 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:55.892464    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85" event={"ID":"f08ef237-f02a-42b4-b935-6abb62d0eaf9","Type":"ContainerStarted","Data":"545de202508ff39c14ebe5b26e2170ec910f8a509de77d0a9644bebdaa2b7f7e"}
Jul 30 00:44:55.892922 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:55.892519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85" event={"ID":"f08ef237-f02a-42b4-b935-6abb62d0eaf9","Type":"ContainerStarted","Data":"d6112481a121a7bdac679ea79b3b959d5f3c2a355603d79e6d6549acd7e2a153"}
Jul 30 00:44:55.909274 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:55.909215    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-h4z85" podStartSLOduration=2.26255507 podStartE2EDuration="3.909191334s" totalImagesPullingTime="1.646636264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:55.907976883 +0000 UTC m=+83.098925215" watchObservedRunningTime="2026-07-30 00:44:55.909191334 +0000 UTC m=+83.100139666"
Jul 30 00:44:58.142905 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.142866    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"]
Jul 30 00:44:58.150078 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.149886    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.153506 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.153309    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-29vxt\""
Jul 30 00:44:58.153506 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.153470    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 30 00:44:58.153714 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.153671    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 30 00:44:58.163780 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.163714    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"]
Jul 30 00:44:58.166451 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.165090    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7fqcv"]
Jul 30 00:44:58.168839 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.168819    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.171159 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.171138    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 00:44:58.171417 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.171402    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 00:44:58.171613 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.171599    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 00:44:58.172157 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.171775    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjmm2\""
Jul 30 00:44:58.304743 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304700    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.304941 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-textfile\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.304941 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304833    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.304941 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304877    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwdbc\" (UniqueName: \"kubernetes.io/projected/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-kube-api-access-jwdbc\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.304941 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-sys\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.304941 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304930    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-root\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.304978    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-wtmp\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305009    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-metrics-client-ca\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60524341-05a3-4fee-8847-6e354d55b589-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.305207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.305186    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx54q\" (UniqueName: \"kubernetes.io/projected/60524341-05a3-4fee-8847-6e354d55b589-kube-api-access-gx54q\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.406180 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-metrics-client-ca\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406180 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406162    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60524341-05a3-4fee-8847-6e354d55b589-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406232    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx54q\" (UniqueName: \"kubernetes.io/projected/60524341-05a3-4fee-8847-6e354d55b589-kube-api-access-gx54q\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406331    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-textfile\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406383 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406716 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406403    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwdbc\" (UniqueName: \"kubernetes.io/projected/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-kube-api-access-jwdbc\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406716 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406429    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-sys\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406716 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-root\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406716 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-wtmp\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.406716 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407006 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.406816    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-metrics-client-ca\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407006 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:58.406968    2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 30 00:44:58.407108 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:44:58.407027    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls podName:8c9e072d-42ce-4fc5-8a5a-2b91f2842f06 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:58.907009088 +0000 UTC m=+86.097957400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls") pod "node-exporter-7fqcv" (UID: "8c9e072d-42ce-4fc5-8a5a-2b91f2842f06") : secret "node-exporter-tls" not found
Jul 30 00:44:58.407108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407039    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60524341-05a3-4fee-8847-6e354d55b589-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.407108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407070    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-root\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407089    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407097    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-textfile\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407365 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407114    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-sys\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.407365 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.407230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-wtmp\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.409712 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.409692    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.409912 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.409867    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.409912 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.409887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60524341-05a3-4fee-8847-6e354d55b589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.414936 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.414904    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwdbc\" (UniqueName: \"kubernetes.io/projected/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-kube-api-access-jwdbc\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.415485 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.415464    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx54q\" (UniqueName: \"kubernetes.io/projected/60524341-05a3-4fee-8847-6e354d55b589-kube-api-access-gx54q\") pod \"openshift-state-metrics-b8f49f5c6-rtr87\" (UID: \"60524341-05a3-4fee-8847-6e354d55b589\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.464762 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.464510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"
Jul 30 00:44:58.851078 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.851056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87"]
Jul 30 00:44:58.854263 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:58.854236    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60524341_05a3_4fee_8847_6e354d55b589.slice/crio-9b9ba44f38ace6fc68b6ec203c287264566d04bc7bb841e530c69b5e60c0f5b3 WatchSource:0}: Error finding container 9b9ba44f38ace6fc68b6ec203c287264566d04bc7bb841e530c69b5e60c0f5b3: Status 404 returned error can't find the container with id 9b9ba44f38ace6fc68b6ec203c287264566d04bc7bb841e530c69b5e60c0f5b3
Jul 30 00:44:58.906887 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.906850    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tjr6x" event={"ID":"fe11ab37-4387-4566-ae2c-3febc2522285","Type":"ContainerStarted","Data":"b150224f11eb5d4c116462d51550719be4dd1557923867fcfad9fd4b8b512b76"}
Jul 30 00:44:58.908460 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.908436    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87" event={"ID":"60524341-05a3-4fee-8847-6e354d55b589","Type":"ContainerStarted","Data":"9b9ba44f38ace6fc68b6ec203c287264566d04bc7bb841e530c69b5e60c0f5b3"}
Jul 30 00:44:58.910579 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.910552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:58.912789 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:58.912770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c9e072d-42ce-4fc5-8a5a-2b91f2842f06-node-exporter-tls\") pod \"node-exporter-7fqcv\" (UID: \"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06\") " pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:59.082173 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:59.082137    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7fqcv"
Jul 30 00:44:59.093649 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:44:59.093611    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9e072d_42ce_4fc5_8a5a_2b91f2842f06.slice/crio-8e969817952bcc7a76f77b313a29d5b5553ae1ee10c4cca7aeb0a6a33064219e WatchSource:0}: Error finding container 8e969817952bcc7a76f77b313a29d5b5553ae1ee10c4cca7aeb0a6a33064219e: Status 404 returned error can't find the container with id 8e969817952bcc7a76f77b313a29d5b5553ae1ee10c4cca7aeb0a6a33064219e
Jul 30 00:44:59.918818 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:59.917896    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87" event={"ID":"60524341-05a3-4fee-8847-6e354d55b589","Type":"ContainerStarted","Data":"89ddc25cfc29b49f6bcfb803e3269c1bbc5cc499d935dcf9dfd2359e337b01d2"}
Jul 30 00:44:59.918818 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:59.917975    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87" event={"ID":"60524341-05a3-4fee-8847-6e354d55b589","Type":"ContainerStarted","Data":"8b629155945cddffcf6afe369fc6f657ede633f1ec4856cce5bd9f25fa6aa071"}
Jul 30 00:44:59.920587 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:44:59.920557    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fqcv" event={"ID":"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06","Type":"ContainerStarted","Data":"8e969817952bcc7a76f77b313a29d5b5553ae1ee10c4cca7aeb0a6a33064219e"}
Jul 30 00:45:00.926130 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:00.926039    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87" event={"ID":"60524341-05a3-4fee-8847-6e354d55b589","Type":"ContainerStarted","Data":"71b75aca2ea13fbc506ddc29cf9e481ccb4b86bdc94e40a6bb20ad36693e35a5"}
Jul 30 00:45:00.927991 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:00.927960    2584 generic.go:356] "Generic (PLEG): container finished" podID="8c9e072d-42ce-4fc5-8a5a-2b91f2842f06" containerID="3372cfa60e592ea59b4a5826aeb8ee9f6486ae83e0ca26315f0c401437dc2524" exitCode=0
Jul 30 00:45:00.928119 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:00.928000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fqcv" event={"ID":"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06","Type":"ContainerDied","Data":"3372cfa60e592ea59b4a5826aeb8ee9f6486ae83e0ca26315f0c401437dc2524"}
Jul 30 00:45:00.944310 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:00.944265    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rtr87" podStartSLOduration=1.347305992 podStartE2EDuration="2.944248829s" totalImagesPullingTime="1.596942837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:00.943926752 +0000 UTC m=+88.134875084" watchObservedRunningTime="2026-07-30 00:45:00.944248829 +0000 UTC m=+88.135197160"
Jul 30 00:45:02.938536 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:02.938495    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fqcv" event={"ID":"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06","Type":"ContainerStarted","Data":"fa15115ba9cf09ac8696541f5178162d267efa95c3e266554e2f758fc0a8472f"}
Jul 30 00:45:02.938980 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:02.938561    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fqcv" event={"ID":"8c9e072d-42ce-4fc5-8a5a-2b91f2842f06","Type":"ContainerStarted","Data":"a23ae2d9eb5358beaf075d6bc58d877231589917d83663652e4dd8b08fadd345"}
Jul 30 00:45:02.942717 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:02.942690    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tjr6x" event={"ID":"fe11ab37-4387-4566-ae2c-3febc2522285","Type":"ContainerStarted","Data":"908405245f09e66f9c1f8d0fb7ce29f925151e562d18e6a7c092da96a4de8560"}
Jul 30 00:45:02.959890 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:02.959836    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7fqcv" podStartSLOduration=3.413885758 podStartE2EDuration="4.959815763s" totalImagesPullingTime="1.472549586s" totalInitContainerRuntime="73.380419ms" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:02.958432355 +0000 UTC m=+90.149380684" watchObservedRunningTime="2026-07-30 00:45:02.959815763 +0000 UTC m=+90.150764087"
Jul 30 00:45:04.369250 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.369215    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:45:04.373734 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.373704    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.376382 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.376357    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4i9tu39lt8mqp\""
Jul 30 00:45:04.376551 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.376359    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 00:45:04.377139 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377115    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 00:45:04.377139 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377131    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 00:45:04.377379 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377361    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 00:45:04.377575 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377559    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 00:45:04.377658 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377587    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 00:45:04.377658 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377629    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 00:45:04.377658 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 00:45:04.377840 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n2gpl\""
Jul 30 00:45:04.377840 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377721    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 00:45:04.377840 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377591    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 00:45:04.377990 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.377932    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 00:45:04.378130 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.378111    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 00:45:04.378252 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.378151    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 00:45:04.378252 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.378163    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 00:45:04.379644 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.379621    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 00:45:04.388050 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.387938    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:45:04.463285 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463246    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463466 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463305    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463466 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463466 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463466 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463432    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcb22\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463495    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463600    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463680 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463666    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463724    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463747    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463774    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463829    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.463937 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463932    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.464217 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.463995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.464217 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.464023    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564457 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564457 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb22\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564696 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564486    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564696 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564506    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564831 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564741    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564831 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564787    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564943 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564943 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564866    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564943 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564943 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.564943 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564942    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.564971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565162 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565440 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565440 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565248    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565440 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565440 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565315    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565440 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565304    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.565985 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.566129 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.565987    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.567526 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.567434    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.568133 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.568107    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.568338 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.568315    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.568819 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.568498    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.568819 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.568591    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.569728 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.569194    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.569728 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.569394    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.570124 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.569947    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.570935 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.570908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.571034 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.570959    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.571099 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.571080    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.571205 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.571171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.571555 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.571537    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.572518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.572495    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.572961 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.572942    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.584986 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.584946    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcb22\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22\") pod \"prometheus-k8s-0\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.687137 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.687049    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:04.846887 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:04.846859    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:45:05.846307 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:05.846272    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fc6ddfd74-lbz2m"
Jul 30 00:45:06.540535 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:45:06.540497    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5db2485_5492_4f92_af19_6a6f773e784f.slice/crio-0d7b0f6bdfaeaabddeb9999fe62a9b567a3dc647584551075ac8a5af17c53fbf WatchSource:0}: Error finding container 0d7b0f6bdfaeaabddeb9999fe62a9b567a3dc647584551075ac8a5af17c53fbf: Status 404 returned error can't find the container with id 0d7b0f6bdfaeaabddeb9999fe62a9b567a3dc647584551075ac8a5af17c53fbf
Jul 30 00:45:06.960546 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:06.960440    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tjr6x" event={"ID":"fe11ab37-4387-4566-ae2c-3febc2522285","Type":"ContainerStarted","Data":"2d23799db108290b26455b0985c3bcfce8d19d60ba8fcb252b6b045d678c9505"}
Jul 30 00:45:06.961633 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:06.961610    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"0d7b0f6bdfaeaabddeb9999fe62a9b567a3dc647584551075ac8a5af17c53fbf"}
Jul 30 00:45:06.978957 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:06.978917    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tjr6x" podStartSLOduration=32.84208357 podStartE2EDuration="45.978902628s" totalImagesPullingTime="13.136819058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:06.978234777 +0000 UTC m=+94.169183257" watchObservedRunningTime="2026-07-30 00:45:06.978902628 +0000 UTC m=+94.169850958"
Jul 30 00:45:08.970288 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:08.970257    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c" exitCode=0
Jul 30 00:45:08.970657 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:08.970315    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c"}
Jul 30 00:45:12.988042 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:12.988007    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"}
Jul 30 00:45:12.988424 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:12.988074    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"}
Jul 30 00:45:14.836238 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:14.836211    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gxgks"
Jul 30 00:45:14.999224 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:14.999195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"}
Jul 30 00:45:14.999385 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:14.999257    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"}
Jul 30 00:45:14.999385 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:14.999284    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"}
Jul 30 00:45:14.999385 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:14.999309    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerStarted","Data":"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"}
Jul 30 00:45:15.025636 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:15.025583    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.982207956 podStartE2EDuration="11.025566432s" totalImagesPullingTime="7.024868957s" totalInitContainerRuntime="18.489519ms" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:15.023196756 +0000 UTC m=+102.214145087" watchObservedRunningTime="2026-07-30 00:45:15.025566432 +0000 UTC m=+102.216514763"
Jul 30 00:45:19.687285 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:19.687250    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:45:33.066648 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:33.066616    2584 generic.go:356] "Generic (PLEG): container finished" podID="cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de" containerID="26e5af4c24cace43d1742eabf7ccd109dd218fde1d5a5c5089fce3935884fddb" exitCode=0
Jul 30 00:45:33.067047 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:33.066695    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp" event={"ID":"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de","Type":"ContainerDied","Data":"26e5af4c24cace43d1742eabf7ccd109dd218fde1d5a5c5089fce3935884fddb"}
Jul 30 00:45:33.067047 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:33.067032    2584 scope.go:117] "RemoveContainer" containerID="26e5af4c24cace43d1742eabf7ccd109dd218fde1d5a5c5089fce3935884fddb"
Jul 30 00:45:34.072526 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:45:34.072490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp" event={"ID":"cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de","Type":"ContainerStarted","Data":"6cf9478f51c6a489c22bb069401be47d0a11b83431a98b55188987a559438504"}
Jul 30 00:46:04.687647 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:04.687609    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:04.707700 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:04.707673    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:05.210312 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:05.210281    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:22.706676 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.706637    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.706941    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="prometheus" containerID="cri-o://856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04" gracePeriod=600
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.706984    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy" containerID="cri-o://a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1" gracePeriod=600
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.707006    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="config-reloader" containerID="cri-o://6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429" gracePeriod=600
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.707035    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-web" containerID="cri-o://d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8" gracePeriod=600
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.707072    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21" gracePeriod=600
Jul 30 00:46:22.707181 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:22.706984    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="thanos-sidecar" containerID="cri-o://b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473" gracePeriod=600
Jul 30 00:46:23.262809 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262763    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21" exitCode=0
Jul 30 00:46:23.262809 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262789    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1" exitCode=0
Jul 30 00:46:23.262809 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262811    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473" exitCode=0
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262820    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429" exitCode=0
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262830    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04" exitCode=0
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"}
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262906    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"}
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262932    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"}
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"}
Jul 30 00:46:23.263038 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.262982    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"}
Jul 30 00:46:23.961756 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:23.961729    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.033590 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033509    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033590 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033549    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcb22\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033590 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033572    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033598    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033619    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033645    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033667    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033689    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033748    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033773    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033810    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.033875 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033853    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033884    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033912    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033936    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033963    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.033999    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034023    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034016    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034028    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.034322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034256    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.034775 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034680    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.035013 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.034987    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.035720 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.035695    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f5db2485-5492-4f92-af19-6a6f773e784f\" (UID: \"f5db2485-5492-4f92-af19-6a6f773e784f\") "
Jul 30 00:46:24.036125 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036105    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.036315 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036291    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-metrics-client-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.036427 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036318    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.036427 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036335    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.036427 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.035731    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:46:24.036427 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036087    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.036743 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036718    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.036964 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.036934    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22" (OuterVolumeSpecName: "kube-api-access-wcb22") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "kube-api-access-wcb22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:46:24.037357 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.037306    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:46:24.038268 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.038165    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.038268 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.038230    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.038518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.038475    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.039300 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.039243    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:46:24.039300 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.039287    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config" (OuterVolumeSpecName: "config") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.039763 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.039739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.039909 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.039755    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.040092 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.040060    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out" (OuterVolumeSpecName: "config-out") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:46:24.040883 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.040860    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.041082 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.041059    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.048444 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.048412    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config" (OuterVolumeSpecName: "web-config") pod "f5db2485-5492-4f92-af19-6a6f773e784f" (UID: "f5db2485-5492-4f92-af19-6a6f773e784f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:46:24.137264 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137226    2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-kube-rbac-proxy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137264 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137258    2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137264 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137269    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-db\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137280    2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-web-config\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137290    2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-grpc-tls\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137298    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137307    2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-metrics-client-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137316    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137324    2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5db2485-5492-4f92-af19-6a6f773e784f-config-out\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137333    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137340    2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-config\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137348    2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-tls-assets\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137357    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137365    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5db2485-5492-4f92-af19-6a6f773e784f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137373    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5db2485-5492-4f92-af19-6a6f773e784f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.137488 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.137382    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcb22\" (UniqueName: \"kubernetes.io/projected/f5db2485-5492-4f92-af19-6a6f773e784f-kube-api-access-wcb22\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:46:24.270179 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.270148    2584 generic.go:356] "Generic (PLEG): container finished" podID="f5db2485-5492-4f92-af19-6a6f773e784f" containerID="d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8" exitCode=0
Jul 30 00:46:24.270340 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.270231    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.270340 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.270228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"}
Jul 30 00:46:24.270451 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.270419    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5db2485-5492-4f92-af19-6a6f773e784f","Type":"ContainerDied","Data":"0d7b0f6bdfaeaabddeb9999fe62a9b567a3dc647584551075ac8a5af17c53fbf"}
Jul 30 00:46:24.270495 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.270474    2584 scope.go:117] "RemoveContainer" containerID="6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"
Jul 30 00:46:24.278595 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.278570    2584 scope.go:117] "RemoveContainer" containerID="a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"
Jul 30 00:46:24.285160 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.285144    2584 scope.go:117] "RemoveContainer" containerID="d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"
Jul 30 00:46:24.291461 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.291444    2584 scope.go:117] "RemoveContainer" containerID="b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"
Jul 30 00:46:24.292473 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.292363    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:24.297449 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.297427    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:24.298337 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.298307    2584 scope.go:117] "RemoveContainer" containerID="6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"
Jul 30 00:46:24.304717 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.304694    2584 scope.go:117] "RemoveContainer" containerID="856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"
Jul 30 00:46:24.311894 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.311875    2584 scope.go:117] "RemoveContainer" containerID="56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c"
Jul 30 00:46:24.318244 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.318228    2584 scope.go:117] "RemoveContainer" containerID="6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"
Jul 30 00:46:24.318540 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.318521    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21\": container with ID starting with 6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21 not found: ID does not exist" containerID="6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"
Jul 30 00:46:24.318592 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.318548    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21"} err="failed to get container status \"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21\": rpc error: code = NotFound desc = could not find container \"6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21\": container with ID starting with 6f4df068b1ea5a0eca904c3e08447dd1acea5eeda2113da2cf762ee9ce9e3e21 not found: ID does not exist"
Jul 30 00:46:24.318592 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.318577    2584 scope.go:117] "RemoveContainer" containerID="a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"
Jul 30 00:46:24.318833 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.318814    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1\": container with ID starting with a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1 not found: ID does not exist" containerID="a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"
Jul 30 00:46:24.318879 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.318840    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1"} err="failed to get container status \"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1\": rpc error: code = NotFound desc = could not find container \"a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1\": container with ID starting with a9b7393a9e29247596b28c00b18b864ee363f9f65d2205f476ab3c8613ec12b1 not found: ID does not exist"
Jul 30 00:46:24.318879 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.318858    2584 scope.go:117] "RemoveContainer" containerID="d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"
Jul 30 00:46:24.319099 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.319082    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8\": container with ID starting with d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8 not found: ID does not exist" containerID="d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"
Jul 30 00:46:24.319136 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319103    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8"} err="failed to get container status \"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8\": rpc error: code = NotFound desc = could not find container \"d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8\": container with ID starting with d473bc58b06af1af930b737094c4afae77469d26b229d5d8169453691ac2e1b8 not found: ID does not exist"
Jul 30 00:46:24.319136 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319117    2584 scope.go:117] "RemoveContainer" containerID="b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"
Jul 30 00:46:24.319336 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.319322    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473\": container with ID starting with b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473 not found: ID does not exist" containerID="b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"
Jul 30 00:46:24.319376 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319339    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473"} err="failed to get container status \"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473\": rpc error: code = NotFound desc = could not find container \"b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473\": container with ID starting with b4b0299294ef702f37bfde386e86de6e90c1fb203b83c04205d4d13b18624473 not found: ID does not exist"
Jul 30 00:46:24.319376 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319350    2584 scope.go:117] "RemoveContainer" containerID="6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"
Jul 30 00:46:24.319548 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.319532    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429\": container with ID starting with 6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429 not found: ID does not exist" containerID="6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"
Jul 30 00:46:24.319587 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319553    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429"} err="failed to get container status \"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429\": rpc error: code = NotFound desc = could not find container \"6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429\": container with ID starting with 6cc424a846d0b227e5694741e5e267e0f88d8611c103663a5568548556d9d429 not found: ID does not exist"
Jul 30 00:46:24.319587 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319566    2584 scope.go:117] "RemoveContainer" containerID="856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"
Jul 30 00:46:24.319762 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.319748    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04\": container with ID starting with 856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04 not found: ID does not exist" containerID="856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"
Jul 30 00:46:24.319823 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319764    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04"} err="failed to get container status \"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04\": rpc error: code = NotFound desc = could not find container \"856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04\": container with ID starting with 856ddc5c4626f06cb588c23691b42bbfb375f4abef0ece5433362760dfa15e04 not found: ID does not exist"
Jul 30 00:46:24.319823 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.319775    2584 scope.go:117] "RemoveContainer" containerID="56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c"
Jul 30 00:46:24.320074 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:46:24.320056    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c\": container with ID starting with 56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c not found: ID does not exist" containerID="56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c"
Jul 30 00:46:24.320127 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.320077    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c"} err="failed to get container status \"56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c\": rpc error: code = NotFound desc = could not find container \"56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c\": container with ID starting with 56ae247f5530fc7071d5fbf7387e0b6c13c6fc5a67af4bfd2878ebe8a5c1f27c not found: ID does not exist"
Jul 30 00:46:24.327659 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327639    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:24.327946 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327934    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-web"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327949    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-web"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327961    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327972    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327980    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-thanos"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327985    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-thanos"
Jul 30 00:46:24.327996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327994    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="prometheus"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.327999    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="prometheus"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328009    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="init-config-reloader"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328013    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="init-config-reloader"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328025    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="config-reloader"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328029    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="config-reloader"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328036    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="thanos-sidecar"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328045    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="thanos-sidecar"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328088    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="config-reloader"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328096    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-thanos"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328102    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy-web"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328108    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="thanos-sidecar"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328112    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="kube-rbac-proxy"
Jul 30 00:46:24.328163 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.328121    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" containerName="prometheus"
Jul 30 00:46:24.331610 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.331594    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.334186 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.334165    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 00:46:24.334758 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.334456    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 00:46:24.334758 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.334559    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4i9tu39lt8mqp\""
Jul 30 00:46:24.334758 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.334567    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 00:46:24.334758 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.334664    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 00:46:24.335411 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.335245    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 00:46:24.336518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336490    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 00:46:24.336862 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336683    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n2gpl\""
Jul 30 00:46:24.336960 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336870    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 00:46:24.336960 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336905    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 00:46:24.336960 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336905    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 00:46:24.337099 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.336960    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 00:46:24.337309 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.337292    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 00:46:24.337363 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.337316    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 00:46:24.337363 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.337328    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 00:46:24.339388 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.339356    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 00:46:24.342191 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.342172    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 00:46:24.347551 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.347531    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:24.440050 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440014    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440050 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440051    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440074    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440181    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440204    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440221    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440265 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440235    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6ps\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-kube-api-access-kb6ps\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440305    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440413    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440482 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440491    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440537    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440557    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440609 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440583    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440751 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440618    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.440751 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.440659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541762 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541762 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541762 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541993 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541993 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541993 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541993 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541945    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.541993 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.541980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542039    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542073    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542106    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542130    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542168    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542216    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542289    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6ps\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-kube-api-access-kb6ps\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542296    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.542563 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.542375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.544631 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.544600    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config-out\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.544829    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545059    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545113    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-web-config\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.545402 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545350    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546017 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.545993    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546046    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546218 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546569 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546569 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.546701 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.546553    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.547108 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.547087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.547591 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.547573    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.547979 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.547958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.548109 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.548092    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.554472 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.554450    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6ps\" (UniqueName: \"kubernetes.io/projected/70ea5d4f-dbaf-41c1-86cd-f597e66042f0-kube-api-access-kb6ps\") pod \"prometheus-k8s-0\" (UID: \"70ea5d4f-dbaf-41c1-86cd-f597e66042f0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.643873 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.643839    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:46:24.778277 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:24.778208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 00:46:24.781334 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:46:24.781302    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ea5d4f_dbaf_41c1_86cd_f597e66042f0.slice/crio-e05150c00a73ea3fc05650ee9a5b941619c17cd67dc009fec60a636a2edec0e7 WatchSource:0}: Error finding container e05150c00a73ea3fc05650ee9a5b941619c17cd67dc009fec60a636a2edec0e7: Status 404 returned error can't find the container with id e05150c00a73ea3fc05650ee9a5b941619c17cd67dc009fec60a636a2edec0e7
Jul 30 00:46:25.275893 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:25.275860    2584 generic.go:356] "Generic (PLEG): container finished" podID="70ea5d4f-dbaf-41c1-86cd-f597e66042f0" containerID="afceabaf53635234c9059ccf80a05aae108fd1cfa73db66f6ca9f391edd5d4ac" exitCode=0
Jul 30 00:46:25.276293 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:25.275945    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerDied","Data":"afceabaf53635234c9059ccf80a05aae108fd1cfa73db66f6ca9f391edd5d4ac"}
Jul 30 00:46:25.276293 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:25.276019    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"e05150c00a73ea3fc05650ee9a5b941619c17cd67dc009fec60a636a2edec0e7"}
Jul 30 00:46:25.384840 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:25.384787    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5db2485-5492-4f92-af19-6a6f773e784f" path="/var/lib/kubelet/pods/f5db2485-5492-4f92-af19-6a6f773e784f/volumes"
Jul 30 00:46:26.286843 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"9ca65b3cbc95d7b2e458182c9b0f5b14656a691f57c82f9ca2537086ade2789f"}
Jul 30 00:46:26.287187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"0c695e40b929e0333a08a5793a93edc3293b3f0fa79d7dd4b15821ff3d79fa4e"}
Jul 30 00:46:26.287187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286904    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"4b0e20bfb9effd74219f1266d8fcdf603228e5e1a6f69c0dec146a37535aac30"}
Jul 30 00:46:26.287187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286927    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"e8ebfab531e77e75866f480fe77eacedf1a665074011f0d5f5d844282646bcc9"}
Jul 30 00:46:26.287187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286951    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"c7237d30ce3db416fbf7fd61451db25151649aeb0e038fbdd21f323958167bde"}
Jul 30 00:46:26.287187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.286973    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"70ea5d4f-dbaf-41c1-86cd-f597e66042f0","Type":"ContainerStarted","Data":"992443c732de2edb5ea1fc4dbb837b7a04df388d15392f24c2aaabf179b31d00"}
Jul 30 00:46:26.316754 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:26.316617    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.296732783 podStartE2EDuration="2.316595855s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.863072ms" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:26.314255824 +0000 UTC m=+173.505204168" watchObservedRunningTime="2026-07-30 00:46:26.316595855 +0000 UTC m=+173.507544193"
Jul 30 00:46:29.644247 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:46:29.644208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:47:24.644249 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:47:24.644209    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:47:24.659459 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:47:24.659431    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:47:25.513883 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:47:25.513851    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 00:48:33.206735 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:48:33.206704    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 00:49:54.077753 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.077714    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"]
Jul 30 00:49:54.081169 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.081150    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.086070 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.086048    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 00:49:54.086187 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.086144    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 00:49:54.086935 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.086913    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-585w4\""
Jul 30 00:49:54.087030 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.086950    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 00:49:54.087876 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.087853    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 00:49:54.100642 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.100618    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"]
Jul 30 00:49:54.123151 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.123123    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.123250 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.123161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czxbx\" (UniqueName: \"kubernetes.io/projected/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-kube-api-access-czxbx\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.123250 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.123194    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.123319 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.123255    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.123319 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.123285    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224278 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czxbx\" (UniqueName: \"kubernetes.io/projected/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-kube-api-access-czxbx\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224424 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224424 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224411    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224494 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224468    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224576 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224556    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.224751 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.224728    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.226515 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.226492    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.226894 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.226875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.227010 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.226988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.233233 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.233213    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czxbx\" (UniqueName: \"kubernetes.io/projected/5807f19c-c3c2-443e-a7e1-fdbc88553aa0-kube-api-access-czxbx\") pod \"opendatahub-operator-controller-manager-746b4b6684-hlfzf\" (UID: \"5807f19c-c3c2-443e-a7e1-fdbc88553aa0\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.390996 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.390916    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:49:54.526529 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.526402    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"]
Jul 30 00:49:54.528677 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:49:54.528654    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5807f19c_c3c2_443e_a7e1_fdbc88553aa0.slice/crio-0265393c50f2310a602f1db5ebc10ba624a6ae7eb25913c67e364c8ccefa8dca WatchSource:0}: Error finding container 0265393c50f2310a602f1db5ebc10ba624a6ae7eb25913c67e364c8ccefa8dca: Status 404 returned error can't find the container with id 0265393c50f2310a602f1db5ebc10ba624a6ae7eb25913c67e364c8ccefa8dca
Jul 30 00:49:54.530556 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:54.530537    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:49:55.035986 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:55.035953    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf" event={"ID":"5807f19c-c3c2-443e-a7e1-fdbc88553aa0","Type":"ContainerStarted","Data":"0265393c50f2310a602f1db5ebc10ba624a6ae7eb25913c67e364c8ccefa8dca"}
Jul 30 00:49:58.051787 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:58.051748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf" event={"ID":"5807f19c-c3c2-443e-a7e1-fdbc88553aa0","Type":"ContainerStarted","Data":"6f01145537dbfc28b2dbc037365f4d9a19fb303425ac907891f9844ebacc8dbd"}
Jul 30 00:49:59.057351 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:59.057312    2584 generic.go:356] "Generic (PLEG): container finished" podID="5807f19c-c3c2-443e-a7e1-fdbc88553aa0" containerID="6f01145537dbfc28b2dbc037365f4d9a19fb303425ac907891f9844ebacc8dbd" exitCode=0
Jul 30 00:49:59.057717 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:49:59.057373    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf" event={"ID":"5807f19c-c3c2-443e-a7e1-fdbc88553aa0","Type":"ContainerDied","Data":"6f01145537dbfc28b2dbc037365f4d9a19fb303425ac907891f9844ebacc8dbd"}
Jul 30 00:50:00.062915 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:00.062880    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf" event={"ID":"5807f19c-c3c2-443e-a7e1-fdbc88553aa0","Type":"ContainerStarted","Data":"de348f4201d4652800ec6b5b6c7622faf5f4d46b7e329f31561d01306f7dbf6a"}
Jul 30 00:50:00.063311 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:00.062999    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:50:00.084785 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:00.084737    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf" podStartSLOduration=1.70636221 podStartE2EDuration="6.084721142s" totalImagesPullingTime="3.286838145s" totalInitContainerRuntime="1.091520787s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:00.083395752 +0000 UTC m=+387.274344074" watchObservedRunningTime="2026-07-30 00:50:00.084721142 +0000 UTC m=+387.275669472"
Jul 30 00:50:11.069945 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.069910    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-hlfzf"
Jul 30 00:50:11.188449 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.188417    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"]
Jul 30 00:50:11.197966 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.197939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.199438 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.199415    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"]
Jul 30 00:50:11.200917 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.200893    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2rjqc\""
Jul 30 00:50:11.201049 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.200929    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 00:50:11.201049 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.200981    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 00:50:11.201369 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.201347    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 00:50:11.201369 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.201363    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:50:11.201619 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.201602    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 00:50:11.257931 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.257894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-manager-config\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.258092 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.257942    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-metrics-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.258092 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.257961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsl2j\" (UniqueName: \"kubernetes.io/projected/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-kube-api-access-rsl2j\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.258092 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.258037    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.358846 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.358733    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-metrics-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.358846 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.358771    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsl2j\" (UniqueName: \"kubernetes.io/projected/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-kube-api-access-rsl2j\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.359042 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.358974    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.359081 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.359054    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-manager-config\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.359648 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.359623    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-manager-config\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.361207 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.361180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-metrics-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.361298 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.361230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-cert\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.368009 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.367985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsl2j\" (UniqueName: \"kubernetes.io/projected/8b3042e7-0fa9-49cb-955c-6ed694c5a08c-kube-api-access-rsl2j\") pod \"lws-controller-manager-6d674c9db-vvjct\" (UID: \"8b3042e7-0fa9-49cb-955c-6ed694c5a08c\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.507997 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.507954    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:11.632445 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:11.632421    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"]
Jul 30 00:50:11.635029 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:50:11.635002    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3042e7_0fa9_49cb_955c_6ed694c5a08c.slice/crio-b581dbe487e0743dd49bbf3e4919d613d315a41e8c8d7e7f9149b1e5987a2d40 WatchSource:0}: Error finding container b581dbe487e0743dd49bbf3e4919d613d315a41e8c8d7e7f9149b1e5987a2d40: Status 404 returned error can't find the container with id b581dbe487e0743dd49bbf3e4919d613d315a41e8c8d7e7f9149b1e5987a2d40
Jul 30 00:50:12.110526 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:12.110493    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct" event={"ID":"8b3042e7-0fa9-49cb-955c-6ed694c5a08c","Type":"ContainerStarted","Data":"b581dbe487e0743dd49bbf3e4919d613d315a41e8c8d7e7f9149b1e5987a2d40"}
Jul 30 00:50:13.816102 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.816071    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"]
Jul 30 00:50:13.824608 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.824579    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.827190 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.827164    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dqwg4\""
Jul 30 00:50:13.828048 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.828028    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 00:50:13.828159 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.828093    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 00:50:13.828159 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.828098    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 00:50:13.828159 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.828153    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 00:50:13.831581 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.831558    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"]
Jul 30 00:50:13.880585 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.880549    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tls-certs\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.880731 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.880673    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l6fb\" (UniqueName: \"kubernetes.io/projected/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-kube-api-access-9l6fb\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.880790 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.880728    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tmp\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.981606 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.981561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l6fb\" (UniqueName: \"kubernetes.io/projected/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-kube-api-access-9l6fb\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.981777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.981628    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tmp\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.981777 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.981660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tls-certs\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.984120 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.984095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tmp\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.984457 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.984436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-tls-certs\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:13.989421 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:13.989397    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l6fb\" (UniqueName: \"kubernetes.io/projected/f05029d5-2f2f-4cf3-a132-a600cdb23b7d-kube-api-access-9l6fb\") pod \"kube-auth-proxy-95755dfb5-kmw2m\" (UID: \"f05029d5-2f2f-4cf3-a132-a600cdb23b7d\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:14.137626 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:14.137552    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"
Jul 30 00:50:14.408533 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:50:14.408502    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05029d5_2f2f_4cf3_a132_a600cdb23b7d.slice/crio-b1f9b352f72edfd8fab5021a6606c9c2907ef302fef2de580616807e720910df WatchSource:0}: Error finding container b1f9b352f72edfd8fab5021a6606c9c2907ef302fef2de580616807e720910df: Status 404 returned error can't find the container with id b1f9b352f72edfd8fab5021a6606c9c2907ef302fef2de580616807e720910df
Jul 30 00:50:14.409627 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:14.409603    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m"]
Jul 30 00:50:15.129665 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:15.129629    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct" event={"ID":"8b3042e7-0fa9-49cb-955c-6ed694c5a08c","Type":"ContainerStarted","Data":"dd7fd7d91673075b9f7f5194cc951e0d58b1af314238afabbadb2f32253669ef"}
Jul 30 00:50:15.130236 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:15.129834    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:50:15.131332 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:15.131303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m" event={"ID":"f05029d5-2f2f-4cf3-a132-a600cdb23b7d","Type":"ContainerStarted","Data":"b1f9b352f72edfd8fab5021a6606c9c2907ef302fef2de580616807e720910df"}
Jul 30 00:50:15.154518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:15.154468    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct" podStartSLOduration=1.468765608 podStartE2EDuration="4.154452726s" totalImagesPullingTime="2.685687118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:15.154231099 +0000 UTC m=+402.345179431" watchObservedRunningTime="2026-07-30 00:50:15.154452726 +0000 UTC m=+402.345401056"
Jul 30 00:50:18.145419 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:18.145388    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m" event={"ID":"f05029d5-2f2f-4cf3-a132-a600cdb23b7d","Type":"ContainerStarted","Data":"e3f7f45063c05c62b011745da4fe37e8b01aef04f1496db52796649648cf67d2"}
Jul 30 00:50:18.160626 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:18.160581    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-95755dfb5-kmw2m" podStartSLOduration=2.327826781 podStartE2EDuration="5.160567338s" totalImagesPullingTime="2.832740557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:18.159920375 +0000 UTC m=+405.350868709" watchObservedRunningTime="2026-07-30 00:50:18.160567338 +0000 UTC m=+405.351515669"
Jul 30 00:50:26.138332 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:50:26.138296    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-vvjct"
Jul 30 00:52:06.063892 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.063862    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"]
Jul 30 00:52:06.067060 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.067044    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.069622 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.069600    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 00:52:06.070573 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.070546    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 00:52:06.070692 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.070581    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t6b4\""
Jul 30 00:52:06.070692 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.070599    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 00:52:06.070692 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.070599    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 00:52:06.076102 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.076082    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"]
Jul 30 00:52:06.110269 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.110248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2a96e726-75b5-41de-82be-751a40e9e09f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.110364 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.110295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.110364 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.110331    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzb6t\" (UniqueName: \"kubernetes.io/projected/2a96e726-75b5-41de-82be-751a40e9e09f-kube-api-access-rzb6t\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.211518 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.211483    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzb6t\" (UniqueName: \"kubernetes.io/projected/2a96e726-75b5-41de-82be-751a40e9e09f-kube-api-access-rzb6t\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.211625 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.211545    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2a96e726-75b5-41de-82be-751a40e9e09f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.211625 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.211582    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.211693 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:52:06.211666    2584 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 30 00:52:06.211732 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:52:06.211723    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert podName:2a96e726-75b5-41de-82be-751a40e9e09f nodeName:}" failed. No retries permitted until 2026-07-30 00:52:06.711706063 +0000 UTC m=+513.902654373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-jsfc4" (UID: "2a96e726-75b5-41de-82be-751a40e9e09f") : secret "plugin-serving-cert" not found
Jul 30 00:52:06.212352 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.212331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2a96e726-75b5-41de-82be-751a40e9e09f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.219459 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.219434    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzb6t\" (UniqueName: \"kubernetes.io/projected/2a96e726-75b5-41de-82be-751a40e9e09f-kube-api-access-rzb6t\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.716588 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.716552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.718957 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.718933    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a96e726-75b5-41de-82be-751a40e9e09f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jsfc4\" (UID: \"2a96e726-75b5-41de-82be-751a40e9e09f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:06.977930 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:06.977842    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"
Jul 30 00:52:07.103263 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:07.103240    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4"]
Jul 30 00:52:07.105922 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:52:07.105894    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a96e726_75b5_41de_82be_751a40e9e09f.slice/crio-a4a14748ee2a7e27526dd03bf1b766a0316e3c95b56b05e4d652277d632d73a6 WatchSource:0}: Error finding container a4a14748ee2a7e27526dd03bf1b766a0316e3c95b56b05e4d652277d632d73a6: Status 404 returned error can't find the container with id a4a14748ee2a7e27526dd03bf1b766a0316e3c95b56b05e4d652277d632d73a6
Jul 30 00:52:07.601176 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:07.601143    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4" event={"ID":"2a96e726-75b5-41de-82be-751a40e9e09f","Type":"ContainerStarted","Data":"a4a14748ee2a7e27526dd03bf1b766a0316e3c95b56b05e4d652277d632d73a6"}
Jul 30 00:52:29.715629 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:29.715590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4" event={"ID":"2a96e726-75b5-41de-82be-751a40e9e09f","Type":"ContainerStarted","Data":"394e614227d66d5fae698f7c2d9eac8f1170b02e8dc5756170f00a9bffd6baf3"}
Jul 30 00:52:29.734231 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:29.734181    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jsfc4" podStartSLOduration=1.963966381 podStartE2EDuration="23.73416627s" totalImagesPullingTime="21.770199889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:29.733085332 +0000 UTC m=+536.924033666" watchObservedRunningTime="2026-07-30 00:52:29.73416627 +0000 UTC m=+536.925114601"
Jul 30 00:52:47.588069 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.588028    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:52:47.598142 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.598117    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.600660 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.600637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 00:52:47.608195 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.608172    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:52:47.630911 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.630886    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:52:47.665058 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.665034    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjlnj\" (UniqueName: \"kubernetes.io/projected/69b340f6-26de-4eca-920b-d57af1bd7b52-kube-api-access-gjlnj\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.665128 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.665086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69b340f6-26de-4eca-920b-d57af1bd7b52-config-file\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.765509 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.765476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjlnj\" (UniqueName: \"kubernetes.io/projected/69b340f6-26de-4eca-920b-d57af1bd7b52-kube-api-access-gjlnj\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.765656 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.765528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69b340f6-26de-4eca-920b-d57af1bd7b52-config-file\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.766112 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.766095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69b340f6-26de-4eca-920b-d57af1bd7b52-config-file\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.779087 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.779055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjlnj\" (UniqueName: \"kubernetes.io/projected/69b340f6-26de-4eca-920b-d57af1bd7b52-kube-api-access-gjlnj\") pod \"limitador-limitador-54d6d988d5-686rb\" (UID: \"69b340f6-26de-4eca-920b-d57af1bd7b52\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:47.908064 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:47.907990    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:48.034326 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.034304    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:52:48.036632 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:52:48.036600    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b340f6_26de_4eca_920b_d57af1bd7b52.slice/crio-ec719337c49df80d2cf472182836cb9a6f084ae698455e1ac1e9738a5d1aa92f WatchSource:0}: Error finding container ec719337c49df80d2cf472182836cb9a6f084ae698455e1ac1e9738a5d1aa92f: Status 404 returned error can't find the container with id ec719337c49df80d2cf472182836cb9a6f084ae698455e1ac1e9738a5d1aa92f
Jul 30 00:52:48.455866 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.455830    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:52:48.458593 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.458576    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:52:48.461060 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.461043    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q2ck5\""
Jul 30 00:52:48.464486 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.464463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:52:48.572170 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.572140    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lmfr\" (UniqueName: \"kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr\") pod \"authorino-75d556bd77-sww6x\" (UID: \"4076a308-ce95-4a21-a52e-6117aee25b05\") " pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:52:48.672920 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.672884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lmfr\" (UniqueName: \"kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr\") pod \"authorino-75d556bd77-sww6x\" (UID: \"4076a308-ce95-4a21-a52e-6117aee25b05\") " pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:52:48.683747 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.683690    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lmfr\" (UniqueName: \"kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr\") pod \"authorino-75d556bd77-sww6x\" (UID: \"4076a308-ce95-4a21-a52e-6117aee25b05\") " pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:52:48.769534 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.769223    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:52:48.798378 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.798345    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb" event={"ID":"69b340f6-26de-4eca-920b-d57af1bd7b52","Type":"ContainerStarted","Data":"ec719337c49df80d2cf472182836cb9a6f084ae698455e1ac1e9738a5d1aa92f"}
Jul 30 00:52:48.944809 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:48.944738    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:52:48.948046 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:52:48.947961    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4076a308_ce95_4a21_a52e_6117aee25b05.slice/crio-dc6075f11672d34d059631cce13f31a7a3f358b5ace6c31a1770d46f23d60435 WatchSource:0}: Error finding container dc6075f11672d34d059631cce13f31a7a3f358b5ace6c31a1770d46f23d60435: Status 404 returned error can't find the container with id dc6075f11672d34d059631cce13f31a7a3f358b5ace6c31a1770d46f23d60435
Jul 30 00:52:49.804552 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:49.804507    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sww6x" event={"ID":"4076a308-ce95-4a21-a52e-6117aee25b05","Type":"ContainerStarted","Data":"dc6075f11672d34d059631cce13f31a7a3f358b5ace6c31a1770d46f23d60435"}
Jul 30 00:52:50.812022 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:50.811988    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb" event={"ID":"69b340f6-26de-4eca-920b-d57af1bd7b52","Type":"ContainerStarted","Data":"0cf186eb6254408e3e24212fa56fe190f1efcc4fee423674ab88367baf956310"}
Jul 30 00:52:50.812433 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:50.812057    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:52:50.830251 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:50.830210    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb" podStartSLOduration=1.169180836 podStartE2EDuration="3.830191168s" totalImagesPullingTime="2.661010332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:50.828222272 +0000 UTC m=+558.019170627" watchObservedRunningTime="2026-07-30 00:52:50.830191168 +0000 UTC m=+558.021139500"
Jul 30 00:52:53.826015 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:53.825929    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sww6x" event={"ID":"4076a308-ce95-4a21-a52e-6117aee25b05","Type":"ContainerStarted","Data":"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"}
Jul 30 00:52:53.840968 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:52:53.840911    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-sww6x" podStartSLOduration=1.2599703070000001 podStartE2EDuration="5.840896427s" totalImagesPullingTime="4.58092612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:53.840228087 +0000 UTC m=+561.031176420" watchObservedRunningTime="2026-07-30 00:52:53.840896427 +0000 UTC m=+561.031844757"
Jul 30 00:53:01.817633 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:01.817597    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-686rb"
Jul 30 00:53:16.843837 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:16.843789    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:53:16.871733 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:16.871698    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:53:16.871912 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:16.871787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:16.874300 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:16.874280    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 00:53:17.020240 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.020204    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.020240 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.020244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lzrz\" (UniqueName: \"kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.121573 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.121489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.121573 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.121527    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lzrz\" (UniqueName: \"kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.124050 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.124029    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.131715 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.131677    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lzrz\" (UniqueName: \"kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz\") pod \"authorino-66576f74db-przzn\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") " pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.180503 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.180479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:53:17.309154 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.309121    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:53:17.312562 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:53:17.312535    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85eda068_dc38_4826_b26d_045d7107370b.slice/crio-0cb5893443978dc406cdef724b96567bc57cffe73bcec575b66d665a97c41527 WatchSource:0}: Error finding container 0cb5893443978dc406cdef724b96567bc57cffe73bcec575b66d665a97c41527: Status 404 returned error can't find the container with id 0cb5893443978dc406cdef724b96567bc57cffe73bcec575b66d665a97c41527
Jul 30 00:53:17.930910 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.930829    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66576f74db-przzn" event={"ID":"85eda068-dc38-4826-b26d-045d7107370b","Type":"ContainerStarted","Data":"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"}
Jul 30 00:53:17.930910 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.930881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66576f74db-przzn" event={"ID":"85eda068-dc38-4826-b26d-045d7107370b","Type":"ContainerStarted","Data":"0cb5893443978dc406cdef724b96567bc57cffe73bcec575b66d665a97c41527"}
Jul 30 00:53:17.951649 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.947196    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66576f74db-przzn" podStartSLOduration=1.59147098 podStartE2EDuration="1.947169604s" totalImagesPullingTime="355.698624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:53:17.944602546 +0000 UTC m=+585.135550880" watchObservedRunningTime="2026-07-30 00:53:17.947169604 +0000 UTC m=+585.138117936"
Jul 30 00:53:17.969670 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.969620    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:53:17.969915 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:17.969875    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-sww6x" podUID="4076a308-ce95-4a21-a52e-6117aee25b05" containerName="authorino" containerID="cri-o://1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc" gracePeriod=30
Jul 30 00:53:18.219878 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.219852    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:53:18.331553 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.331517    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lmfr\" (UniqueName: \"kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr\") pod \"4076a308-ce95-4a21-a52e-6117aee25b05\" (UID: \"4076a308-ce95-4a21-a52e-6117aee25b05\") "
Jul 30 00:53:18.333679 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.333654    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr" (OuterVolumeSpecName: "kube-api-access-2lmfr") pod "4076a308-ce95-4a21-a52e-6117aee25b05" (UID: "4076a308-ce95-4a21-a52e-6117aee25b05"). InnerVolumeSpecName "kube-api-access-2lmfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:53:18.432256 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.432214    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lmfr\" (UniqueName: \"kubernetes.io/projected/4076a308-ce95-4a21-a52e-6117aee25b05-kube-api-access-2lmfr\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:53:18.936577 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.936542    2584 generic.go:356] "Generic (PLEG): container finished" podID="4076a308-ce95-4a21-a52e-6117aee25b05" containerID="1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc" exitCode=0
Jul 30 00:53:18.937054 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.936587    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sww6x"
Jul 30 00:53:18.937054 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.936636    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sww6x" event={"ID":"4076a308-ce95-4a21-a52e-6117aee25b05","Type":"ContainerDied","Data":"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"}
Jul 30 00:53:18.937054 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.936687    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sww6x" event={"ID":"4076a308-ce95-4a21-a52e-6117aee25b05","Type":"ContainerDied","Data":"dc6075f11672d34d059631cce13f31a7a3f358b5ace6c31a1770d46f23d60435"}
Jul 30 00:53:18.937054 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.936711    2584 scope.go:117] "RemoveContainer" containerID="1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"
Jul 30 00:53:18.945045 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.945028    2584 scope.go:117] "RemoveContainer" containerID="1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"
Jul 30 00:53:18.945297 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:53:18.945279    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc\": container with ID starting with 1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc not found: ID does not exist" containerID="1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"
Jul 30 00:53:18.945354 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.945310    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc"} err="failed to get container status \"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc\": rpc error: code = NotFound desc = could not find container \"1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc\": container with ID starting with 1a2cabfba0f5e062caa89edb7bdaf9b3f940e801154456f69f92d115da38cbfc not found: ID does not exist"
Jul 30 00:53:18.955812 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.955773    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:53:18.961407 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:18.961388    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sww6x"]
Jul 30 00:53:19.385250 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:53:19.385220    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4076a308-ce95-4a21-a52e-6117aee25b05" path="/var/lib/kubelet/pods/4076a308-ce95-4a21-a52e-6117aee25b05/volumes"
Jul 30 00:54:00.252849 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:54:00.252762    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:54:24.855186 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:54:24.855154    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:54:41.545466 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:54:41.545430    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:54:51.747781 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:54:51.747745    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:55:13.340102 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:13.340059    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:55:29.449906 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:29.449823    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:55:30.333601 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.333558    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"]
Jul 30 00:55:30.334103 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.334081    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4076a308-ce95-4a21-a52e-6117aee25b05" containerName="authorino"
Jul 30 00:55:30.334103 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.334106    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4076a308-ce95-4a21-a52e-6117aee25b05" containerName="authorino"
Jul 30 00:55:30.334260 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.334248    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4076a308-ce95-4a21-a52e-6117aee25b05" containerName="authorino"
Jul 30 00:55:30.338074 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.338046    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.342164 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.342137    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 00:55:30.342370 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.342348    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 00:55:30.342530 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.342511    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 00:55:30.342707 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.342690    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-stf8p\""
Jul 30 00:55:30.350388 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.350340    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"]
Jul 30 00:55:30.505569 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505535    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.505569 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505571    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.506061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.506061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505710    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.506061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505831    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgl5\" (UniqueName: \"kubernetes.io/projected/3fb369e4-79ed-476c-956c-3fca5a4ce041-kube-api-access-6qgl5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.506061 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.505867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb369e4-79ed-476c-956c-3fca5a4ce041-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606402    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606439    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606489 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606469    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgl5\" (UniqueName: \"kubernetes.io/projected/3fb369e4-79ed-476c-956c-3fca5a4ce041-kube-api-access-6qgl5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb369e4-79ed-476c-956c-3fca5a4ce041-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.606963 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.607011 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.606981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.607147 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.607125    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.608924 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.608899    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb369e4-79ed-476c-956c-3fca5a4ce041-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.609527 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.609502    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb369e4-79ed-476c-956c-3fca5a4ce041-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.616146 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.616121    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgl5\" (UniqueName: \"kubernetes.io/projected/3fb369e4-79ed-476c-956c-3fca5a4ce041-kube-api-access-6qgl5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd\" (UID: \"3fb369e4-79ed-476c-956c-3fca5a4ce041\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.650767 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.650725    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:30.784710 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.784662    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"]
Jul 30 00:55:30.787417 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:55:30.787385    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb369e4_79ed_476c_956c_3fca5a4ce041.slice/crio-5cc9862d91470494c26e590712de44b9757f43c457df00dcf9caf56727f926f8 WatchSource:0}: Error finding container 5cc9862d91470494c26e590712de44b9757f43c457df00dcf9caf56727f926f8: Status 404 returned error can't find the container with id 5cc9862d91470494c26e590712de44b9757f43c457df00dcf9caf56727f926f8
Jul 30 00:55:30.788877 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:30.788858    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:55:31.510589 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:31.510556    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd" event={"ID":"3fb369e4-79ed-476c-956c-3fca5a4ce041","Type":"ContainerStarted","Data":"5cc9862d91470494c26e590712de44b9757f43c457df00dcf9caf56727f926f8"}
Jul 30 00:55:32.515832 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:32.515782    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd" event={"ID":"3fb369e4-79ed-476c-956c-3fca5a4ce041","Type":"ContainerStarted","Data":"bed7e18bab5a90376b98b78c79764bb7c3c54fc9a99613998a8c0d7c58f744bd"}
Jul 30 00:55:32.540727 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:32.540667    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd" podStartSLOduration=1.3869595239999999 podStartE2EDuration="2.54064494s" totalImagesPullingTime="1.153685416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:55:32.539747478 +0000 UTC m=+719.730695800" watchObservedRunningTime="2026-07-30 00:55:32.54064494 +0000 UTC m=+719.731593272"
Jul 30 00:55:39.233367 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:39.233326    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:55:40.651150 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:40.651110    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:40.651628 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:40.651173    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:40.663600 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:40.663576    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:55:41.567022 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:55:41.566987    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd"
Jul 30 00:56:23.148615 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.148568    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7545554d8c-7l8hw"]
Jul 30 00:56:23.152073 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.152050    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.159644 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.159624    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545554d8c-7l8hw"]
Jul 30 00:56:23.250303 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.250269    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2de6e24-4bb1-42e2-b164-97bb38507006-tls-cert\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.250478 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.250318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4zwf\" (UniqueName: \"kubernetes.io/projected/d2de6e24-4bb1-42e2-b164-97bb38507006-kube-api-access-g4zwf\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.351772 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.351730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4zwf\" (UniqueName: \"kubernetes.io/projected/d2de6e24-4bb1-42e2-b164-97bb38507006-kube-api-access-g4zwf\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.351968 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.351919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2de6e24-4bb1-42e2-b164-97bb38507006-tls-cert\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.354433 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.354399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2de6e24-4bb1-42e2-b164-97bb38507006-tls-cert\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.359357 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.359333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4zwf\" (UniqueName: \"kubernetes.io/projected/d2de6e24-4bb1-42e2-b164-97bb38507006-kube-api-access-g4zwf\") pod \"authorino-7545554d8c-7l8hw\" (UID: \"d2de6e24-4bb1-42e2-b164-97bb38507006\") " pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.462042 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.461957    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545554d8c-7l8hw"
Jul 30 00:56:23.598104 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.598080    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545554d8c-7l8hw"]
Jul 30 00:56:23.600307 ip-10-0-134-28 kubenswrapper[2584]: W0730 00:56:23.600268    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2de6e24_4bb1_42e2_b164_97bb38507006.slice/crio-734cf6a3adb73eb2b22c8861b926658833053c0762259eadbd6eb837edd24c7d WatchSource:0}: Error finding container 734cf6a3adb73eb2b22c8861b926658833053c0762259eadbd6eb837edd24c7d: Status 404 returned error can't find the container with id 734cf6a3adb73eb2b22c8861b926658833053c0762259eadbd6eb837edd24c7d
Jul 30 00:56:23.737362 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:23.737330    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545554d8c-7l8hw" event={"ID":"d2de6e24-4bb1-42e2-b164-97bb38507006","Type":"ContainerStarted","Data":"734cf6a3adb73eb2b22c8861b926658833053c0762259eadbd6eb837edd24c7d"}
Jul 30 00:56:24.745150 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:24.745108    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545554d8c-7l8hw" event={"ID":"d2de6e24-4bb1-42e2-b164-97bb38507006","Type":"ContainerStarted","Data":"11549ec3d8449d035cd05446f4ae4c9ca11f388c4cd7b259fb2783c2ea2b95e5"}
Jul 30 00:56:24.762322 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:24.762250    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7545554d8c-7l8hw" podStartSLOduration=1.3495859989999999 podStartE2EDuration="1.762230479s" totalImagesPullingTime="412.64448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:56:24.761174724 +0000 UTC m=+771.952123055" watchObservedRunningTime="2026-07-30 00:56:24.762230479 +0000 UTC m=+771.953178813"
Jul 30 00:56:24.798289 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:24.798255    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:56:24.798701 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:24.798672    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66576f74db-przzn" podUID="85eda068-dc38-4826-b26d-045d7107370b" containerName="authorino" containerID="cri-o://895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5" gracePeriod=30
Jul 30 00:56:25.056487 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.056460    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:56:25.169998 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.169966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert\") pod \"85eda068-dc38-4826-b26d-045d7107370b\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") "
Jul 30 00:56:25.170152 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.170020    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lzrz\" (UniqueName: \"kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz\") pod \"85eda068-dc38-4826-b26d-045d7107370b\" (UID: \"85eda068-dc38-4826-b26d-045d7107370b\") "
Jul 30 00:56:25.172107 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.172068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz" (OuterVolumeSpecName: "kube-api-access-8lzrz") pod "85eda068-dc38-4826-b26d-045d7107370b" (UID: "85eda068-dc38-4826-b26d-045d7107370b"). InnerVolumeSpecName "kube-api-access-8lzrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:56:25.179394 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.179366    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "85eda068-dc38-4826-b26d-045d7107370b" (UID: "85eda068-dc38-4826-b26d-045d7107370b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:56:25.270704 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.270669    2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85eda068-dc38-4826-b26d-045d7107370b-tls-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:56:25.270704 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.270701    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lzrz\" (UniqueName: \"kubernetes.io/projected/85eda068-dc38-4826-b26d-045d7107370b-kube-api-access-8lzrz\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 00:56:25.750342 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.750308    2584 generic.go:356] "Generic (PLEG): container finished" podID="85eda068-dc38-4826-b26d-045d7107370b" containerID="895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5" exitCode=0
Jul 30 00:56:25.750825 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.750366    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66576f74db-przzn"
Jul 30 00:56:25.750825 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.750405    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66576f74db-przzn" event={"ID":"85eda068-dc38-4826-b26d-045d7107370b","Type":"ContainerDied","Data":"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"}
Jul 30 00:56:25.750825 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.750460    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66576f74db-przzn" event={"ID":"85eda068-dc38-4826-b26d-045d7107370b","Type":"ContainerDied","Data":"0cb5893443978dc406cdef724b96567bc57cffe73bcec575b66d665a97c41527"}
Jul 30 00:56:25.750825 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.750485    2584 scope.go:117] "RemoveContainer" containerID="895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"
Jul 30 00:56:25.759542 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.759525    2584 scope.go:117] "RemoveContainer" containerID="895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"
Jul 30 00:56:25.759851 ip-10-0-134-28 kubenswrapper[2584]: E0730 00:56:25.759830    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5\": container with ID starting with 895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5 not found: ID does not exist" containerID="895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"
Jul 30 00:56:25.759933 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.759863    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5"} err="failed to get container status \"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5\": rpc error: code = NotFound desc = could not find container \"895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5\": container with ID starting with 895f64192926cba19e7e8271c094d3ce7cb5628a61fed012df8d8433bdb927c5 not found: ID does not exist"
Jul 30 00:56:25.781027 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.780991    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:56:25.790285 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:25.790260    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66576f74db-przzn"]
Jul 30 00:56:27.384999 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:27.384966    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85eda068-dc38-4826-b26d-045d7107370b" path="/var/lib/kubelet/pods/85eda068-dc38-4826-b26d-045d7107370b/volumes"
Jul 30 00:56:52.754209 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:56:52.754169    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:57:02.245154 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:57:02.245062    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:57:13.133675 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:57:13.133635    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:58:58.239734 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:58:58.239691    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 00:59:56.242705 ip-10-0-134-28 kubenswrapper[2584]: I0730 00:59:56.242626    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:00:00.139167 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.139131    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"]
Jul 30 01:00:00.139540 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.139465    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85eda068-dc38-4826-b26d-045d7107370b" containerName="authorino"
Jul 30 01:00:00.139540 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.139476    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="85eda068-dc38-4826-b26d-045d7107370b" containerName="authorino"
Jul 30 01:00:00.139540 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.139531    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="85eda068-dc38-4826-b26d-045d7107370b" containerName="authorino"
Jul 30 01:00:00.142461 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.142445    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"
Jul 30 01:00:00.144899 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.144878    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:00:00.149242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.149220    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"]
Jul 30 01:00:00.153933 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.153914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"
Jul 30 01:00:00.487102 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.487078    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"]
Jul 30 01:00:00.488956 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:00:00.488931    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bf7bad_0b78_4799_8786_58e46c0e3927.slice/crio-ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8 WatchSource:0}: Error finding container ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8: Status 404 returned error can't find the container with id ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8
Jul 30 01:00:00.691245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:00.691209    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69" event={"ID":"46bf7bad-0b78-4799-8786-58e46c0e3927","Type":"ContainerStarted","Data":"ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8"}
Jul 30 01:00:04.711623 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:04.711547    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69" event={"ID":"46bf7bad-0b78-4799-8786-58e46c0e3927","Type":"ContainerStarted","Data":"334baabf433d2bf199eef626a9b4f0d2feecc0802e5d9c07bdbdf1ddede04622"}
Jul 30 01:00:04.725479 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:04.725433    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69" podStartSLOduration=0.852806055 podStartE2EDuration="4.725420234s" totalImagesPullingTime="3.872614179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:00:04.724843447 +0000 UTC m=+991.915791775" watchObservedRunningTime="2026-07-30 01:00:04.725420234 +0000 UTC m=+991.916368565"
Jul 30 01:00:05.717420 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:05.717385    2584 generic.go:356] "Generic (PLEG): container finished" podID="46bf7bad-0b78-4799-8786-58e46c0e3927" containerID="334baabf433d2bf199eef626a9b4f0d2feecc0802e5d9c07bdbdf1ddede04622" exitCode=0
Jul 30 01:00:05.717880 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:05.717434    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69" event={"ID":"46bf7bad-0b78-4799-8786-58e46c0e3927","Type":"ContainerDied","Data":"334baabf433d2bf199eef626a9b4f0d2feecc0802e5d9c07bdbdf1ddede04622"}
Jul 30 01:00:06.542670 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:06.542627    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:00:06.855862 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:06.855836    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"
Jul 30 01:00:07.727558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:07.727477    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69" event={"ID":"46bf7bad-0b78-4799-8786-58e46c0e3927","Type":"ContainerDied","Data":"ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8"}
Jul 30 01:00:07.727558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:07.727528    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba96df991b030265f3bced37106eb9607895a91b7095c4d135fc7faf1f78bcc8"
Jul 30 01:00:07.727558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:07.727489    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"
Jul 30 01:00:21.840850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:21.840818    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:00:30.938892 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:30.938857    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:00:41.138920 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:41.138881    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:00:50.838274 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:00:50.838234    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:01:01.838590 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:01:01.838552    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:01:11.939293 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:01:11.939257    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:01:34.238679 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:01:34.238578    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:01:47.752693 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:01:47.752654    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:02:07.252886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:02:07.252850    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:02:30.546239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:02:30.546205    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:02:45.151899 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:02:45.151856    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:03:05.937726 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:03:05.937647    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:03:21.747750 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:03:21.747705    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:03:37.040664 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:03:37.040628    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:03:51.954900 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:03:51.954839    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:04:00.554203 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:04:00.554157    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:04:22.155048 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:04:22.155015    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:04:32.547117 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:04:32.547041    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:04:48.377301 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:04:48.377265    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:07:01.749746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:07:01.749703    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:07:11.251437 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:07:11.251397    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:07:43.946917 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:07:43.946829    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:07:52.743337 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:07:52.743300    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:08:05.347088 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:08:05.347052    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:08:13.839190 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:08:13.839148    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:08:26.635386 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:08:26.635343    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:08:44.586405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:08:44.586368    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:08:53.736462 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:08:53.736425    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:09:06.446018 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:09:06.445933    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:09:14.242132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:09:14.242096    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:09:34.348458 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:09:34.348418    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:09:42.063064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:09:42.063030    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:10:10.051874 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.051838    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:10:10.052392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.052154    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46bf7bad-0b78-4799-8786-58e46c0e3927" containerName="cleanup"
Jul 30 01:10:10.052392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.052164    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="46bf7bad-0b78-4799-8786-58e46c0e3927" containerName="cleanup"
Jul 30 01:10:10.052392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.052222    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="46bf7bad-0b78-4799-8786-58e46c0e3927" containerName="cleanup"
Jul 30 01:10:10.054225 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.054209    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.056447 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.056427    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-70b5b8ed-kserve-self-signed-certs\""
Jul 30 01:10:10.073666 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.073637    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:10:10.177868 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.177831    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.178027 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.177878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.178027 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.177910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.178027 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.178010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.178146 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.178038    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krpms\" (UniqueName: \"kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.178146 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.178063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279182    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279221    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279254    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279314    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krpms\" (UniqueName: \"kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279371    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279690 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279692    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.279787 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.279749    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.281589 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.281564    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.281818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.281784    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.287870 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.287850    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krpms\" (UniqueName: \"kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms\") pod \"e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") " pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.364101 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.364021    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:10.495489 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:10.495461    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:10:10.497179 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:10:10.497146    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf753ce3d_78ad_4775_9f78_7e002b8d92a7.slice/crio-9127d4dec0e3a4d7bdbd5b33b9c21e5d5986113ba8caa75c516345514e4467a7 WatchSource:0}: Error finding container 9127d4dec0e3a4d7bdbd5b33b9c21e5d5986113ba8caa75c516345514e4467a7: Status 404 returned error can't find the container with id 9127d4dec0e3a4d7bdbd5b33b9c21e5d5986113ba8caa75c516345514e4467a7
Jul 30 01:10:11.346789 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:11.346748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" event={"ID":"f753ce3d-78ad-4775-9f78-7e002b8d92a7","Type":"ContainerStarted","Data":"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"}
Jul 30 01:10:11.347198 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:11.346791    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" event={"ID":"f753ce3d-78ad-4775-9f78-7e002b8d92a7","Type":"ContainerStarted","Data":"9127d4dec0e3a4d7bdbd5b33b9c21e5d5986113ba8caa75c516345514e4467a7"}
Jul 30 01:10:11.364171 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:11.364117    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" podStartSLOduration=1.3640994100000001 podStartE2EDuration="1.36409941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:10:11.362710677 +0000 UTC m=+1598.553659000" watchObservedRunningTime="2026-07-30 01:10:11.36409941 +0000 UTC m=+1598.555047743"
Jul 30 01:10:13.648518 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:13.648481    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:10:20.364219 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:20.364120    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:20.364606 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:20.364213    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:20.377057 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:20.377033    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:20.396163 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:20.396138    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:10:28.240328 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.240239    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:10:28.242753 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.242730    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.245220 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.245199    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-70b5b8ed-kserve-self-signed-certs\""
Jul 30 01:10:28.252940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.252917    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:10:28.326366 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326331    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.326366 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.326636 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.326636 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326559    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.326636 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326595    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfdv6\" (UniqueName: \"kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.326636 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.326629    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427347 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427310    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427535 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427610 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427610 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427565    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427610 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdv6\" (UniqueName: \"kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427755    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.427778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.428017 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.427992    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.429946 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.429923    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.430083 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.430064    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.444162 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.444130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdv6\" (UniqueName: \"kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6\") pod \"e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") " pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.553019 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.552938    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:28.686262 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:28.686236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:10:28.687688 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:10:28.687659    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod289c9f68_7095_4bd1_861c_4fd156d1fc0f.slice/crio-d9c4e75c28e929fa567e032651ca1334e93f58110b866598ec32bcb676aac3ab WatchSource:0}: Error finding container d9c4e75c28e929fa567e032651ca1334e93f58110b866598ec32bcb676aac3ab: Status 404 returned error can't find the container with id d9c4e75c28e929fa567e032651ca1334e93f58110b866598ec32bcb676aac3ab
Jul 30 01:10:29.425770 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:29.425732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" event={"ID":"289c9f68-7095-4bd1-861c-4fd156d1fc0f","Type":"ContainerStarted","Data":"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"}
Jul 30 01:10:29.425770 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:29.425777    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" event={"ID":"289c9f68-7095-4bd1-861c-4fd156d1fc0f","Type":"ContainerStarted","Data":"d9c4e75c28e929fa567e032651ca1334e93f58110b866598ec32bcb676aac3ab"}
Jul 30 01:10:29.442650 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:29.442599    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" podStartSLOduration=1.442582797 podStartE2EDuration="1.442582797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:10:29.44231234 +0000 UTC m=+1616.633260672" watchObservedRunningTime="2026-07-30 01:10:29.442582797 +0000 UTC m=+1616.633531128"
Jul 30 01:10:32.342449 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:32.342402    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:10:38.553304 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:38.553264    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:38.553827 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:38.553319    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:38.566118 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:38.566090    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:39.482303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:39.482273    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:10:49.604321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:49.604289    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:10:50.729059 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:50.729023    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:10:54.052575 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:10:54.052529    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:11:04.623258 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:04.623217    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" podUID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" containerName="main" containerID="cri-o://8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2" gracePeriod=45
Jul 30 01:11:04.863345 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:04.863320    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:11:05.056079 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056040    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056079 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056084    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krpms\" (UniqueName: \"kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056319 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056131    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056319 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056153    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056318    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056399    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm\") pod \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\" (UID: \"f753ce3d-78ad-4775-9f78-7e002b8d92a7\") "
Jul 30 01:11:05.056521 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056469    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home" (OuterVolumeSpecName: "home") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:05.056521 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056482    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:05.056624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056597    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:05.056754 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056731    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.056754 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056755    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.056892 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.056764    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.058513 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.058482    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:11:05.058625 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.058518    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm" (OuterVolumeSpecName: "dshm") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:05.058625 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.058528    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms" (OuterVolumeSpecName: "kube-api-access-krpms") pod "f753ce3d-78ad-4775-9f78-7e002b8d92a7" (UID: "f753ce3d-78ad-4775-9f78-7e002b8d92a7"). InnerVolumeSpecName "kube-api-access-krpms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:11:05.157314 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.157269    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f753ce3d-78ad-4775-9f78-7e002b8d92a7-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.157314 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.157310    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f753ce3d-78ad-4775-9f78-7e002b8d92a7-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.157314 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.157321    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krpms\" (UniqueName: \"kubernetes.io/projected/f753ce3d-78ad-4775-9f78-7e002b8d92a7-kube-api-access-krpms\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:05.589815 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.589763    2584 generic.go:356] "Generic (PLEG): container finished" podID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" containerID="8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2" exitCode=0
Jul 30 01:11:05.590004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.589823    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"
Jul 30 01:11:05.590004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.589850    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" event={"ID":"f753ce3d-78ad-4775-9f78-7e002b8d92a7","Type":"ContainerDied","Data":"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"}
Jul 30 01:11:05.590004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.589899    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj" event={"ID":"f753ce3d-78ad-4775-9f78-7e002b8d92a7","Type":"ContainerDied","Data":"9127d4dec0e3a4d7bdbd5b33b9c21e5d5986113ba8caa75c516345514e4467a7"}
Jul 30 01:11:05.590004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.589921    2584 scope.go:117] "RemoveContainer" containerID="8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"
Jul 30 01:11:05.598138 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.598121    2584 scope.go:117] "RemoveContainer" containerID="8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"
Jul 30 01:11:05.598404 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:11:05.598385    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2\": container with ID starting with 8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2 not found: ID does not exist" containerID="8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"
Jul 30 01:11:05.598447 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.598412    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2"} err="failed to get container status \"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2\": rpc error: code = NotFound desc = could not find container \"8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2\": container with ID starting with 8ba3cc17cbe6b9cfd04fd19f705d4c4a663ff17022c8410abd9fccc77eae04b2 not found: ID does not exist"
Jul 30 01:11:05.627019 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.626617    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:11:05.630790 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.630764    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-70b5b8ed-kserve-54cfccffb7-pq2wj"]
Jul 30 01:11:05.747690 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.747648    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" podUID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" containerName="main" containerID="cri-o://9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056" gracePeriod=45
Jul 30 01:11:05.996224 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:05.996198    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:11:06.065778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065698    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.065778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065731    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.065778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065751    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.065778 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065769    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.066127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065839    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfdv6\" (UniqueName: \"kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.066127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.065914    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs\") pod \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\" (UID: \"289c9f68-7095-4bd1-861c-4fd156d1fc0f\") "
Jul 30 01:11:06.066127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066031    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:06.066127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066067    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache" (OuterVolumeSpecName: "model-cache") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:06.066127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066109    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home" (OuterVolumeSpecName: "home") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:06.066360 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066266    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.066360 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066286    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.066360 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.066301    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.068000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.067979    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:11:06.068000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.067991    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6" (OuterVolumeSpecName: "kube-api-access-lfdv6") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "kube-api-access-lfdv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:11:06.068108 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.068063    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm" (OuterVolumeSpecName: "dshm") pod "289c9f68-7095-4bd1-861c-4fd156d1fc0f" (UID: "289c9f68-7095-4bd1-861c-4fd156d1fc0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:11:06.167211 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.167180    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/289c9f68-7095-4bd1-861c-4fd156d1fc0f-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.167211 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.167207    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/289c9f68-7095-4bd1-861c-4fd156d1fc0f-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.167211 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.167220    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfdv6\" (UniqueName: \"kubernetes.io/projected/289c9f68-7095-4bd1-861c-4fd156d1fc0f-kube-api-access-lfdv6\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:11:06.596775 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.596741    2584 generic.go:356] "Generic (PLEG): container finished" podID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" containerID="9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056" exitCode=0
Jul 30 01:11:06.596961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.596781    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"
Jul 30 01:11:06.596961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.596829    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" event={"ID":"289c9f68-7095-4bd1-861c-4fd156d1fc0f","Type":"ContainerDied","Data":"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"}
Jul 30 01:11:06.596961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.596870    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l" event={"ID":"289c9f68-7095-4bd1-861c-4fd156d1fc0f","Type":"ContainerDied","Data":"d9c4e75c28e929fa567e032651ca1334e93f58110b866598ec32bcb676aac3ab"}
Jul 30 01:11:06.596961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.596887    2584 scope.go:117] "RemoveContainer" containerID="9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"
Jul 30 01:11:06.605659 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.605644    2584 scope.go:117] "RemoveContainer" containerID="9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"
Jul 30 01:11:06.605937 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:11:06.605919    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056\": container with ID starting with 9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056 not found: ID does not exist" containerID="9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"
Jul 30 01:11:06.606024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.605944    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056"} err="failed to get container status \"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056\": rpc error: code = NotFound desc = could not find container \"9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056\": container with ID starting with 9c167d24da45a9400b0ccae2440b5998ba36a7daee7e0e302275627f2127c056 not found: ID does not exist"
Jul 30 01:11:06.619729 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.619705    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:11:06.624809 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:06.624775    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-70b5b8ed-kserve-7454bd569f-zp77l"]
Jul 30 01:11:07.386263 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:07.386232    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" path="/var/lib/kubelet/pods/289c9f68-7095-4bd1-861c-4fd156d1fc0f/volumes"
Jul 30 01:11:07.386651 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:07.386637    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" path="/var/lib/kubelet/pods/f753ce3d-78ad-4775-9f78-7e002b8d92a7/volumes"
Jul 30 01:11:07.951766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:07.951734    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:11:19.150158 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:19.150122    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:11:31.564417 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:31.564385    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:11:47.203120 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:47.203082    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:11:58.842309 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:11:58.842233    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:07.140547 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:07.140511    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:16.251641 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:16.251604    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:26.042637 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:26.042600    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:39.840881 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:39.840847    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:48.343160 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:48.343121    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:12:58.647580 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:12:58.647544    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:13:11.644879 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:13:11.644844    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:13:23.041910 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:13:23.041873    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:13:31.242467 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:13:31.242432    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:13:53.840562 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:13:53.840527    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:14:06.745879 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:14:06.745842    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:14:21.337789 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:14:21.337748    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:14:36.345885 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:14:36.345853    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:14:45.639017 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:14:45.638981    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:14:59.037372 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:14:59.037293    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:15:00.135259 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135228    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"]
Jul 30 01:15:00.135628 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135540    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" containerName="main"
Jul 30 01:15:00.135628 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135552    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" containerName="main"
Jul 30 01:15:00.135628 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135571    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" containerName="main"
Jul 30 01:15:00.135628 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135577    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" containerName="main"
Jul 30 01:15:00.135628 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135623    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="289c9f68-7095-4bd1-861c-4fd156d1fc0f" containerName="main"
Jul 30 01:15:00.135786 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.135633    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f753ce3d-78ad-4775-9f78-7e002b8d92a7" containerName="main"
Jul 30 01:15:00.141632 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.141615    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"
Jul 30 01:15:00.144155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.144135    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:15:00.150742 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.150724    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"
Jul 30 01:15:00.153490 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.153463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"]
Jul 30 01:15:00.278183 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.278160    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"]
Jul 30 01:15:00.280451 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:15:00.280418    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44afff0b_34b1_4283_ace6_0b6e3049d5f0.slice/crio-d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb WatchSource:0}: Error finding container d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb: Status 404 returned error can't find the container with id d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb
Jul 30 01:15:00.642997 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.642962    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r" event={"ID":"44afff0b-34b1-4283-ace6-0b6e3049d5f0","Type":"ContainerStarted","Data":"5b97106ab6ac5233d01ea68bfb1beceacc6982c55a8afc6d79a2d63f8f914967"}
Jul 30 01:15:00.643197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.643018    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r" event={"ID":"44afff0b-34b1-4283-ace6-0b6e3049d5f0","Type":"ContainerStarted","Data":"d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb"}
Jul 30 01:15:00.659211 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:00.659166    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r" podStartSLOduration=0.65915294 podStartE2EDuration="659.15294ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:15:00.658013707 +0000 UTC m=+1887.848962041" watchObservedRunningTime="2026-07-30 01:15:00.65915294 +0000 UTC m=+1887.850101332"
Jul 30 01:15:01.648667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:01.648634    2584 generic.go:356] "Generic (PLEG): container finished" podID="44afff0b-34b1-4283-ace6-0b6e3049d5f0" containerID="5b97106ab6ac5233d01ea68bfb1beceacc6982c55a8afc6d79a2d63f8f914967" exitCode=0
Jul 30 01:15:01.649065 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:01.648716    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r" event={"ID":"44afff0b-34b1-4283-ace6-0b6e3049d5f0","Type":"ContainerDied","Data":"5b97106ab6ac5233d01ea68bfb1beceacc6982c55a8afc6d79a2d63f8f914967"}
Jul 30 01:15:02.778984 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:02.778960    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"
Jul 30 01:15:03.658976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:03.658887    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r" event={"ID":"44afff0b-34b1-4283-ace6-0b6e3049d5f0","Type":"ContainerDied","Data":"d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb"}
Jul 30 01:15:03.658976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:03.658945    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8f1260eead6d475059118c6e7fa589c6329892aced71368e7f6c5275a25ccbb"
Jul 30 01:15:03.658976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:03.658899    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756235-2qb9r"
Jul 30 01:15:13.243555 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:13.243524    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:15:23.453317 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:23.453276    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:15:33.268688 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:33.268654    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:15:43.850617 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:15:43.850584    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:16:01.644666 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.644628    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"]
Jul 30 01:16:01.645048 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.644985    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44afff0b-34b1-4283-ace6-0b6e3049d5f0" containerName="cleanup"
Jul 30 01:16:01.645048 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.644998    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="44afff0b-34b1-4283-ace6-0b6e3049d5f0" containerName="cleanup"
Jul 30 01:16:01.645123 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.645076    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="44afff0b-34b1-4283-ace6-0b6e3049d5f0" containerName="cleanup"
Jul 30 01:16:01.648335 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.648316    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:01.653247 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.653227    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:16:01.653530 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.653514    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:16:01.664178 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.664158    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"]
Jul 30 01:16:01.821507 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.821474    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn\" (UID: \"9c1c7e62-5ab1-45f4-aec2-a088074e7761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:01.922560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.922482    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn\" (UID: \"9c1c7e62-5ab1-45f4-aec2-a088074e7761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:01.923117 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.923098    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn\" (UID: \"9c1c7e62-5ab1-45f4-aec2-a088074e7761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:01.958567 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:01.958536    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:02.089108 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:02.089086    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"]
Jul 30 01:16:02.091226 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:16:02.091201    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1c7e62_5ab1_45f4_aec2_a088074e7761.slice/crio-3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826 WatchSource:0}: Error finding container 3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826: Status 404 returned error can't find the container with id 3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826
Jul 30 01:16:02.918074 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:02.918043    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c1c7e62-5ab1-45f4-aec2-a088074e7761" containerID="e40c43b402ecd700cd23260a57f668377f6f7ce33010830f5635ac01d8224f5f" exitCode=0
Jul 30 01:16:02.918432 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:02.918113    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn" event={"ID":"9c1c7e62-5ab1-45f4-aec2-a088074e7761","Type":"ContainerDied","Data":"e40c43b402ecd700cd23260a57f668377f6f7ce33010830f5635ac01d8224f5f"}
Jul 30 01:16:02.918432 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:02.918171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn" event={"ID":"9c1c7e62-5ab1-45f4-aec2-a088074e7761","Type":"ContainerStarted","Data":"3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826"}
Jul 30 01:16:04.046593 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.046571    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:04.144270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.144239    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca\") pod \"9c1c7e62-5ab1-45f4-aec2-a088074e7761\" (UID: \"9c1c7e62-5ab1-45f4-aec2-a088074e7761\") "
Jul 30 01:16:04.144602 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.144581    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c1c7e62-5ab1-45f4-aec2-a088074e7761" (UID: "9c1c7e62-5ab1-45f4-aec2-a088074e7761"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:04.245711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.245679    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1c7e62-5ab1-45f4-aec2-a088074e7761-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:16:04.928055 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.928018    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn" event={"ID":"9c1c7e62-5ab1-45f4-aec2-a088074e7761","Type":"ContainerDied","Data":"3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826"}
Jul 30 01:16:04.928055 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.928045    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"
Jul 30 01:16:04.928255 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:04.928097    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e899b6b4553d0e00391a6fd2bbe69f8370f7cc4940a7c92c14180f052cac826"
Jul 30 01:16:07.877221 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:07.877186    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"]
Jul 30 01:16:07.878704 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:07.878684    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-76aaf06c-f67vn"]
Jul 30 01:16:09.385558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:09.385524    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c1c7e62-5ab1-45f4-aec2-a088074e7761" path="/var/lib/kubelet/pods/9c1c7e62-5ab1-45f4-aec2-a088074e7761/volumes"
Jul 30 01:16:11.950345 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:11.950305    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:16:22.646858 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:22.646823    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:16:36.338661 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.338580    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"]
Jul 30 01:16:36.339103 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.338947    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c1c7e62-5ab1-45f4-aec2-a088074e7761" containerName="revoke-keys"
Jul 30 01:16:36.339103 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.338958    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c1c7e62-5ab1-45f4-aec2-a088074e7761" containerName="revoke-keys"
Jul 30 01:16:36.339103 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.339017    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c1c7e62-5ab1-45f4-aec2-a088074e7761" containerName="revoke-keys"
Jul 30 01:16:36.341819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.341784    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:36.344423 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.344398    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:16:36.344561 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.344440    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:16:36.347954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.347927    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"]
Jul 30 01:16:36.410432 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.410396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt\" (UID: \"7ee3919e-9718-4f1d-87f9-e3425c2455ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:36.511736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.511704    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt\" (UID: \"7ee3919e-9718-4f1d-87f9-e3425c2455ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:36.512366 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.512345    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt\" (UID: \"7ee3919e-9718-4f1d-87f9-e3425c2455ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:36.654086 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.654004    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:36.781118 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:36.781083    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"]
Jul 30 01:16:36.784427 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:16:36.784397    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee3919e_9718_4f1d_87f9_e3425c2455ed.slice/crio-1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30 WatchSource:0}: Error finding container 1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30: Status 404 returned error can't find the container with id 1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30
Jul 30 01:16:37.071380 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:37.071344    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt" event={"ID":"7ee3919e-9718-4f1d-87f9-e3425c2455ed","Type":"ContainerStarted","Data":"1f30b824b85477d888937b76710e778c93c05356028f8f742423ac41e79241fc"}
Jul 30 01:16:37.071547 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:37.071404    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt" event={"ID":"7ee3919e-9718-4f1d-87f9-e3425c2455ed","Type":"ContainerStarted","Data":"1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30"}
Jul 30 01:16:37.087435 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:37.087384    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt" podStartSLOduration=1.087369682 podStartE2EDuration="1.087369682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:37.085542619 +0000 UTC m=+1984.276490944" watchObservedRunningTime="2026-07-30 01:16:37.087369682 +0000 UTC m=+1984.278317991"
Jul 30 01:16:38.077708 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:38.077664    2584 generic.go:356] "Generic (PLEG): container finished" podID="7ee3919e-9718-4f1d-87f9-e3425c2455ed" containerID="1f30b824b85477d888937b76710e778c93c05356028f8f742423ac41e79241fc" exitCode=0
Jul 30 01:16:38.077708 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:38.077694    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt" event={"ID":"7ee3919e-9718-4f1d-87f9-e3425c2455ed","Type":"ContainerDied","Data":"1f30b824b85477d888937b76710e778c93c05356028f8f742423ac41e79241fc"}
Jul 30 01:16:39.207772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:39.207747    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:39.338179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:39.338090    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca\") pod \"7ee3919e-9718-4f1d-87f9-e3425c2455ed\" (UID: \"7ee3919e-9718-4f1d-87f9-e3425c2455ed\") "
Jul 30 01:16:39.338444 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:39.338421    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ee3919e-9718-4f1d-87f9-e3425c2455ed" (UID: "7ee3919e-9718-4f1d-87f9-e3425c2455ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:39.439138 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:39.439102    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee3919e-9718-4f1d-87f9-e3425c2455ed-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:16:40.093605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:40.093573    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"
Jul 30 01:16:40.093605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:40.093587    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt" event={"ID":"7ee3919e-9718-4f1d-87f9-e3425c2455ed","Type":"ContainerDied","Data":"1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30"}
Jul 30 01:16:40.093819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:40.093657    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b084302e93d79e39dba234a83163932ad6c5f429564689f7acf972666c42b30"
Jul 30 01:16:42.983409 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:42.983372    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"]
Jul 30 01:16:42.989067 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:42.989040    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a6605e44-92kxt"]
Jul 30 01:16:43.385872 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.385842    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ee3919e-9718-4f1d-87f9-e3425c2455ed" path="/var/lib/kubelet/pods/7ee3919e-9718-4f1d-87f9-e3425c2455ed/volumes"
Jul 30 01:16:43.882983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.882951    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:16:43.883277 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.883264    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ee3919e-9718-4f1d-87f9-e3425c2455ed" containerName="revoke-keys"
Jul 30 01:16:43.883277 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.883279    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ee3919e-9718-4f1d-87f9-e3425c2455ed" containerName="revoke-keys"
Jul 30 01:16:43.883395 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.883344    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ee3919e-9718-4f1d-87f9-e3425c2455ed" containerName="revoke-keys"
Jul 30 01:16:43.886415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.886395    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.888868 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.888787    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:16:43.888954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.888819    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-095ac0a0-openshift-default-dockercfg-7m2z8\""
Jul 30 01:16:43.898227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.898206    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:16:43.982667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.982897 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.982897 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982744    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.982897 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982840    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.982897 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982880    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djlls\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.983110 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982933    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.983110 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982964    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.983110 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.982996    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:43.983110 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:43.983071    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.083687 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083652    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djlls\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083719    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083842    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083911    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.083946    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084001    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084073    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084148    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084286    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084433    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084501    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.084544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.084511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.086511 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.086479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.086746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.086723    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.094595 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.094568    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djlls\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.094904 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.094881    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token\") pod \"e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") " pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.198657 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.198580    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:44.339870 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.339843    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:16:44.341215 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:16:44.341184    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9619ae_52a7_40e1_9d75_de07b432b5a8.slice/crio-3ac2ded2047b92f36bded50509f102520caf73a6dd57f81c5292ff70bcf00768 WatchSource:0}: Error finding container 3ac2ded2047b92f36bded50509f102520caf73a6dd57f81c5292ff70bcf00768: Status 404 returned error can't find the container with id 3ac2ded2047b92f36bded50509f102520caf73a6dd57f81c5292ff70bcf00768
Jul 30 01:16:44.343168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:44.343152    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:16:45.119480 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:45.119442    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerStarted","Data":"3ac2ded2047b92f36bded50509f102520caf73a6dd57f81c5292ff70bcf00768"}
Jul 30 01:16:46.844695 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:46.844652    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:16:46.845089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:46.844750    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:16:46.845089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:46.844826    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:16:47.046655 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:47.046618    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:16:47.131695 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:47.131604    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerStarted","Data":"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"}
Jul 30 01:16:47.153328 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:47.153275    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podStartSLOduration=1.652233897 podStartE2EDuration="4.153260521s" totalImagesPullingTime="2.501026624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:47.151628062 +0000 UTC m=+1994.342576393" watchObservedRunningTime="2026-07-30 01:16:47.153260521 +0000 UTC m=+1994.344208855"
Jul 30 01:16:47.199403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:47.199370    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:47.204356 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:47.204330    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:48.136495 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:48.136463    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:48.137458 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:48.137437    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:16:59.945166 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:16:59.945126    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:17:02.202290 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.202256    2584 generic.go:356] "Generic (PLEG): container finished" podID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerID="c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c" exitCode=0
Jul 30 01:17:02.202731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.202329    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerDied","Data":"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"}
Jul 30 01:17:02.202731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.202704    2584 scope.go:117] "RemoveContainer" containerID="c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"
Jul 30 01:17:02.204649 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.204618    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:02.204814 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.204704    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:02.204814 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:02.204754    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:03.137166 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:03.137125    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:03.211808 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:03.211752    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerStarted","Data":"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"}
Jul 30 01:17:04.199148 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:04.199110    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:04.200184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:04.200155    2584 patch_prober.go:28] interesting pod/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 01:17:04.200321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:04.200205    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 01:17:05.200022 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:05.199985    2584 patch_prober.go:28] interesting pod/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 01:17:05.200430 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:05.200045    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 01:17:06.199740 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:06.199697    2584 patch_prober.go:28] interesting pod/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 01:17:06.199942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:06.199756    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 01:17:07.203407 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:07.203372    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:07.203876 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:07.203653    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:07.204411 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:07.204394    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:14.287975 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.287939    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"]
Jul 30 01:17:14.293032 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.293008    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:14.295569 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.295545    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:17:14.296063 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.296043    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:17:14.297710 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.297681    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"]
Jul 30 01:17:14.347126 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.347085    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv\" (UID: \"04d18319-e52c-4d45-8602-9339c3974f67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:14.448354 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.448318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv\" (UID: \"04d18319-e52c-4d45-8602-9339c3974f67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:14.448974 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.448953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv\" (UID: \"04d18319-e52c-4d45-8602-9339c3974f67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:14.606202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.606109    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:14.736470 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:14.736445    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"]
Jul 30 01:17:14.738630 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:17:14.738604    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d18319_e52c_4d45_8602_9339c3974f67.slice/crio-2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519 WatchSource:0}: Error finding container 2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519: Status 404 returned error can't find the container with id 2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519
Jul 30 01:17:15.281632 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:15.281600    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv" event={"ID":"04d18319-e52c-4d45-8602-9339c3974f67","Type":"ContainerStarted","Data":"11df90e156046ef96a0dcd9cc9b71ce197f9b013dd7b5aa65e5454eff143f1b5"}
Jul 30 01:17:15.281829 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:15.281650    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv" event={"ID":"04d18319-e52c-4d45-8602-9339c3974f67","Type":"ContainerStarted","Data":"2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519"}
Jul 30 01:17:15.298729 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:15.298683    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv" podStartSLOduration=1.298667968 podStartE2EDuration="1.298667968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:15.297290278 +0000 UTC m=+2022.488238612" watchObservedRunningTime="2026-07-30 01:17:15.298667968 +0000 UTC m=+2022.489616298"
Jul 30 01:17:16.287351 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:16.287317    2584 generic.go:356] "Generic (PLEG): container finished" podID="04d18319-e52c-4d45-8602-9339c3974f67" containerID="11df90e156046ef96a0dcd9cc9b71ce197f9b013dd7b5aa65e5454eff143f1b5" exitCode=0
Jul 30 01:17:16.287558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:16.287405    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv" event={"ID":"04d18319-e52c-4d45-8602-9339c3974f67","Type":"ContainerDied","Data":"11df90e156046ef96a0dcd9cc9b71ce197f9b013dd7b5aa65e5454eff143f1b5"}
Jul 30 01:17:17.426756 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:17.426729    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:17.475720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:17.475687    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca\") pod \"04d18319-e52c-4d45-8602-9339c3974f67\" (UID: \"04d18319-e52c-4d45-8602-9339c3974f67\") "
Jul 30 01:17:17.476018 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:17.475995    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04d18319-e52c-4d45-8602-9339c3974f67" (UID: "04d18319-e52c-4d45-8602-9339c3974f67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:17.576972 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:17.576880    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d18319-e52c-4d45-8602-9339c3974f67-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:18.298596 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:18.298568    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"
Jul 30 01:17:18.298756 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:18.298568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv" event={"ID":"04d18319-e52c-4d45-8602-9339c3974f67","Type":"ContainerDied","Data":"2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519"}
Jul 30 01:17:18.298756 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:18.298693    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2230aca6260e9ff5d0b030e8b9b7df89caab9a04eea3333260a0fbcd7ed2f519"
Jul 30 01:17:20.490656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:20.490617    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"]
Jul 30 01:17:20.493169 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:20.493142    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-095ac0a0-ph2zv"]
Jul 30 01:17:21.386243 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:21.386209    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d18319-e52c-4d45-8602-9339c3974f67" path="/var/lib/kubelet/pods/04d18319-e52c-4d45-8602-9339c3974f67/volumes"
Jul 30 01:17:21.600718 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:21.600681    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:17:21.601157 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:21.600900    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy" containerID="cri-o://3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154" gracePeriod=30
Jul 30 01:17:22.206596 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.206553    2584 patch_prober.go:28] interesting pod/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 30 01:17:22.206777 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.206622    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 01:17:22.335775 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.335732    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:17:22.336111 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.336097    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d18319-e52c-4d45-8602-9339c3974f67" containerName="revoke-keys"
Jul 30 01:17:22.336111 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.336112    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d18319-e52c-4d45-8602-9339c3974f67" containerName="revoke-keys"
Jul 30 01:17:22.336197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.336163    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d18319-e52c-4d45-8602-9339c3974f67" containerName="revoke-keys"
Jul 30 01:17:22.339203 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.339187    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.341692 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.341665    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-14be608d-openshift-default-dockercfg-j9698\""
Jul 30 01:17:22.352093 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.352067    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:17:22.417867 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.417830    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.417867 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.417867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418038 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.417890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418038 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.417943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418038 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.417987    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5xdg\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418169 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.418154    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418205 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.418179    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.418202    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.418239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.418232    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.519664 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.519859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.519859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.519859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.519859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519848    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5xdg\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520083 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.519914    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520083 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520097    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520098    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520123    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520364 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520364 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520200    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520480 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520387    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.520623 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.520605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.522387 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.522364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.522604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.522585    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.528447 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.528426    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5xdg\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.528549 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.528529    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token\") pod \"e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") " pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.649633 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.649602    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:22.781559 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.781232    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:17:22.784576 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:17:22.784552    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5f18a7_f167_40cc_91d8_32e5c7c4ae93.slice/crio-0de06c3a55f632d96ed20f2641eab6de335210d4ab7fc4aef7874c860f56e370 WatchSource:0}: Error finding container 0de06c3a55f632d96ed20f2641eab6de335210d4ab7fc4aef7874c860f56e370: Status 404 returned error can't find the container with id 0de06c3a55f632d96ed20f2641eab6de335210d4ab7fc4aef7874c860f56e370
Jul 30 01:17:22.787082 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.787041    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:22.787162 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.787133    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:22.787202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:22.787190    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:23.325203 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:23.325162    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerStarted","Data":"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"}
Jul 30 01:17:23.325390 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:23.325268    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerStarted","Data":"0de06c3a55f632d96ed20f2641eab6de335210d4ab7fc4aef7874c860f56e370"}
Jul 30 01:17:23.349975 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:23.349917    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podStartSLOduration=1.349900763 podStartE2EDuration="1.349900763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:23.348649613 +0000 UTC m=+2030.539597946" watchObservedRunningTime="2026-07-30 01:17:23.349900763 +0000 UTC m=+2030.540849071"
Jul 30 01:17:23.650290 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:23.650207    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:23.655032 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:23.655003    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:24.330468 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:24.330436    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:24.331476 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:24.331452    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:24.337501 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:24.337477    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:17:26.844067 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.844045    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:26.962180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962085    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djlls\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962130    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962158    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962183    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962267    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962309    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962348    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962371    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962421    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs\") pod \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\" (UID: \"5a9619ae-52a7-40e1-9d75-de07b432b5a8\") "
Jul 30 01:17:26.962488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962445    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:17:26.962792 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:17:26.962792 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962762    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:26.962792 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962781    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:26.962983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.962825    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:26.963049 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.963025    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:17:26.963184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.963163    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:17:26.964662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.964635    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:17:26.964780 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.964661    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls" (OuterVolumeSpecName: "kube-api-access-djlls") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "kube-api-access-djlls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:26.964860 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.964848    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:26.964959 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:26.964942    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a9619ae-52a7-40e1-9d75-de07b432b5a8" (UID: "5a9619ae-52a7-40e1-9d75-de07b432b5a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:17:27.063275 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063241    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djlls\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-kube-api-access-djlls\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063275 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063268    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063275 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063277    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063287    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063295    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063304    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a9619ae-52a7-40e1-9d75-de07b432b5a8-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.063493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.063312    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a9619ae-52a7-40e1-9d75-de07b432b5a8-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:27.347606 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.347563    2584 generic.go:356] "Generic (PLEG): container finished" podID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerID="3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154" exitCode=0
Jul 30 01:17:27.347771 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.347623    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"
Jul 30 01:17:27.347771 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.347616    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerDied","Data":"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"}
Jul 30 01:17:27.347942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.347768    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8" event={"ID":"5a9619ae-52a7-40e1-9d75-de07b432b5a8","Type":"ContainerDied","Data":"3ac2ded2047b92f36bded50509f102520caf73a6dd57f81c5292ff70bcf00768"}
Jul 30 01:17:27.347942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.347814    2584 scope.go:117] "RemoveContainer" containerID="3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"
Jul 30 01:17:27.364871 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.364849    2584 scope.go:117] "RemoveContainer" containerID="c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"
Jul 30 01:17:27.373308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.373290    2584 scope.go:117] "RemoveContainer" containerID="3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"
Jul 30 01:17:27.373477 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.373457    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:17:27.373563 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:17:27.373541    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154\": container with ID starting with 3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154 not found: ID does not exist" containerID="3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"
Jul 30 01:17:27.373602 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.373572    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154"} err="failed to get container status \"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154\": rpc error: code = NotFound desc = could not find container \"3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154\": container with ID starting with 3a11802de8ce9fe39a540a6984c715387d8ae7e6186f97a8f3840bd18492f154 not found: ID does not exist"
Jul 30 01:17:27.373602 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.373591    2584 scope.go:117] "RemoveContainer" containerID="c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"
Jul 30 01:17:27.373789 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:17:27.373774    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c\": container with ID starting with c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c not found: ID does not exist" containerID="c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"
Jul 30 01:17:27.373864 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.373810    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c"} err="failed to get container status \"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c\": rpc error: code = NotFound desc = could not find container \"c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c\": container with ID starting with c71ac1cb47e8c0d94a0612f3ca3c78d4f936ed47cd5da7c8a6aaa0f7b9bdaf1c not found: ID does not exist"
Jul 30 01:17:27.377681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.377662    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-095ac0a0-openshift-default-75cfb7c474-q6tc8"]
Jul 30 01:17:27.385313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:27.385292    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" path="/var/lib/kubelet/pods/5a9619ae-52a7-40e1-9d75-de07b432b5a8/volumes"
Jul 30 01:17:34.744558 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:34.744520    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:17:39.332611 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.332574    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 01:17:39.333101 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.332629    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 01:17:39.405591 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.405556    2584 generic.go:356] "Generic (PLEG): container finished" podID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerID="0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920" exitCode=0
Jul 30 01:17:39.405751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.405632    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerDied","Data":"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"}
Jul 30 01:17:39.406021 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.406008    2584 scope.go:117] "RemoveContainer" containerID="0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"
Jul 30 01:17:39.406945 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.406922    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:39.406997 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.406984    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:39.407034 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:39.407014    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:17:40.415766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:40.415718    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerStarted","Data":"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"}
Jul 30 01:17:40.650324 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:40.650283    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:40.650508 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:40.650483    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 01:17:40.650556 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:40.650541    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 01:17:41.650734 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:41.650690    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 01:17:41.651130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:41.650756    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 01:17:42.650871 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:42.650832    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 01:17:42.651270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:42.650890    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 01:17:43.650202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:43.650164    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 01:17:43.650457 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:43.650222    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 01:17:44.654777 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:44.654748    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:44.655229 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:44.655164    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:44.655888 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:44.655868    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:17:49.549699 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.549667    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"]
Jul 30 01:17:49.550099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550008    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.550099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550020    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.550099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550036    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.550099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550042    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.550235 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550103    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.550235 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.550112    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a9619ae-52a7-40e1-9d75-de07b432b5a8" containerName="istio-proxy"
Jul 30 01:17:49.553288 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.553271    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:49.555866 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.555843    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:17:49.555992 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.555894    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:17:49.559472 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.559445    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"]
Jul 30 01:17:49.674578 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.674541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-14be608d-qkss8\" (UID: \"ad86641f-5892-4c96-8321-48192cd31d63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:49.775214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.775181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-14be608d-qkss8\" (UID: \"ad86641f-5892-4c96-8321-48192cd31d63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:49.775774 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.775754    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-14be608d-qkss8\" (UID: \"ad86641f-5892-4c96-8321-48192cd31d63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:49.864253 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.864193    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:49.990313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:49.990288    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"]
Jul 30 01:17:49.993055 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:17:49.993016    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad86641f_5892_4c96_8321_48192cd31d63.slice/crio-df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe WatchSource:0}: Error finding container df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe: Status 404 returned error can't find the container with id df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe
Jul 30 01:17:50.468958 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:50.468922    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8" event={"ID":"ad86641f-5892-4c96-8321-48192cd31d63","Type":"ContainerStarted","Data":"403e973b39b3b3a5c124f590ad1c8dbd5fd295df767d557093eb3e4f5c044d52"}
Jul 30 01:17:50.469203 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:50.468984    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8" event={"ID":"ad86641f-5892-4c96-8321-48192cd31d63","Type":"ContainerStarted","Data":"df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe"}
Jul 30 01:17:50.484482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:50.484436    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8" podStartSLOduration=1.484420273 podStartE2EDuration="1.484420273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:50.482611994 +0000 UTC m=+2057.673560326" watchObservedRunningTime="2026-07-30 01:17:50.484420273 +0000 UTC m=+2057.675368604"
Jul 30 01:17:51.475456 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:51.475416    2584 generic.go:356] "Generic (PLEG): container finished" podID="ad86641f-5892-4c96-8321-48192cd31d63" containerID="403e973b39b3b3a5c124f590ad1c8dbd5fd295df767d557093eb3e4f5c044d52" exitCode=0
Jul 30 01:17:51.475876 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:51.475501    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8" event={"ID":"ad86641f-5892-4c96-8321-48192cd31d63","Type":"ContainerDied","Data":"403e973b39b3b3a5c124f590ad1c8dbd5fd295df767d557093eb3e4f5c044d52"}
Jul 30 01:17:52.616388 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:52.616363    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:52.802158 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:52.802121    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca\") pod \"ad86641f-5892-4c96-8321-48192cd31d63\" (UID: \"ad86641f-5892-4c96-8321-48192cd31d63\") "
Jul 30 01:17:52.802493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:52.802466    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad86641f-5892-4c96-8321-48192cd31d63" (UID: "ad86641f-5892-4c96-8321-48192cd31d63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:52.903629 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:52.903599    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad86641f-5892-4c96-8321-48192cd31d63-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:17:53.490381 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:53.490356    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"
Jul 30 01:17:53.490548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:53.490392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8" event={"ID":"ad86641f-5892-4c96-8321-48192cd31d63","Type":"ContainerDied","Data":"df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe"}
Jul 30 01:17:53.490548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:53.490439    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df02bb589a959351ecd1b5a20b54fbac1f85b85fd1975ed5c37bea9186553efe"
Jul 30 01:17:56.676827 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:56.676733    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"]
Jul 30 01:17:56.682837 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:56.682782    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-14be608d-qkss8"]
Jul 30 01:17:56.811083 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:56.811053    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:17:56.811296 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:56.811263    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" containerID="cri-o://f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526" gracePeriod=30
Jul 30 01:17:57.390211 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:57.390177    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad86641f-5892-4c96-8321-48192cd31d63" path="/var/lib/kubelet/pods/ad86641f-5892-4c96-8321-48192cd31d63/volumes"
Jul 30 01:17:59.657924 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:59.657891    2584 patch_prober.go:28] interesting pod/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 01:17:59.658365 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:17:59.657941    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 01:18:00.247401 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:00.247359    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:18:02.068141 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.068117    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:18:02.085537 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085459    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5xdg\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085537 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085492    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085558    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085605    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085635    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085671    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085687    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085705    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.085736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.085728    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo\") pod \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\" (UID: \"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93\") "
Jul 30 01:18:02.086194 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.086011    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:18:02.086751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.086428    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:18:02.086751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.086494    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:18:02.086751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.086683    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:18:02.087013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.086777    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:18:02.088169 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.088134    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg" (OuterVolumeSpecName: "kube-api-access-m5xdg") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "kube-api-access-m5xdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:18:02.088305 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.088266    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:18:02.088305 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.088279    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:18:02.088827 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.088738    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" (UID: "3d5f18a7-f167-40cc-91d8-32e5c7c4ae93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:18:02.186766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186730    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.186766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186761    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.186766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186770    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186778    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186787    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186811    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186819    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186827    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5xdg\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-kube-api-access-m5xdg\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.187004 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.186835    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:18:02.533983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.533940    2584 generic.go:356] "Generic (PLEG): container finished" podID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerID="f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526" exitCode=0
Jul 30 01:18:02.534155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.533996    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"
Jul 30 01:18:02.534155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.534022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerDied","Data":"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"}
Jul 30 01:18:02.534155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.534075    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm" event={"ID":"3d5f18a7-f167-40cc-91d8-32e5c7c4ae93","Type":"ContainerDied","Data":"0de06c3a55f632d96ed20f2641eab6de335210d4ab7fc4aef7874c860f56e370"}
Jul 30 01:18:02.534155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.534100    2584 scope.go:117] "RemoveContainer" containerID="f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"
Jul 30 01:18:02.542672 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.542654    2584 scope.go:117] "RemoveContainer" containerID="0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"
Jul 30 01:18:02.550239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.550220    2584 scope.go:117] "RemoveContainer" containerID="f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"
Jul 30 01:18:02.550474 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:18:02.550456    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526\": container with ID starting with f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526 not found: ID does not exist" containerID="f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"
Jul 30 01:18:02.550539 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.550481    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526"} err="failed to get container status \"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526\": rpc error: code = NotFound desc = could not find container \"f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526\": container with ID starting with f6b4fe3c0d243c0259507140af7b08199ded538b43b1dfaa379ebf59a787f526 not found: ID does not exist"
Jul 30 01:18:02.550539 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.550498    2584 scope.go:117] "RemoveContainer" containerID="0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"
Jul 30 01:18:02.550699 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:18:02.550685    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920\": container with ID starting with 0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920 not found: ID does not exist" containerID="0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"
Jul 30 01:18:02.550747 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.550701    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920"} err="failed to get container status \"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920\": rpc error: code = NotFound desc = could not find container \"0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920\": container with ID starting with 0e09e6ccce183008633e021911ebc0f0ce5c6567c2683b4da2603ab9cb58d920 not found: ID does not exist"
Jul 30 01:18:02.557437 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.557414    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:18:02.561954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:02.561934    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-14be608d-openshift-default-d888c6b44-vz5rm"]
Jul 30 01:18:03.385839 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:03.385790    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" path="/var/lib/kubelet/pods/3d5f18a7-f167-40cc-91d8-32e5c7c4ae93/volumes"
Jul 30 01:18:22.741515 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:18:22.741475    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:19:51.839403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:19:51.839371    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:20:02.956935 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:02.956891    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:20:03.429204 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429167    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:20:03.429509 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429497    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:03.429554 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429511    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:03.429554 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429519    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad86641f-5892-4c96-8321-48192cd31d63" containerName="revoke-keys"
Jul 30 01:20:03.429554 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429525    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad86641f-5892-4c96-8321-48192cd31d63" containerName="revoke-keys"
Jul 30 01:20:03.429651 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429578    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:03.429651 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429585    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:03.429651 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.429593    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad86641f-5892-4c96-8321-48192cd31d63" containerName="revoke-keys"
Jul 30 01:20:03.432734 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.432716    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.435312 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.435287    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:20:03.435463 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.435332    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-14e4ae87-openshift-default-dockercfg-q64wr\""
Jul 30 01:20:03.459275 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.459243    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:20:03.558401 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558595 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558412    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558595 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558433    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qxb\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558595 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558595 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558582    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558609    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.558772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.558748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659688    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659767    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659832    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qxb\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.659999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.659972    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.660303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.660000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.660303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.660072    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.660533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.660505    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.660885 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.660862    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.661186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.661159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.661328 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.661302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.661634 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.661610    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.663163 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.663141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.663593 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.663571    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.668392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.668369    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.668654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.668636    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qxb\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb\") pod \"e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") " pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.747399 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.747365    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:03.906671 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.906556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:20:03.909115 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:20:03.909085    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd14620f_2965_4a45_bc61_3ae50ac95dff.slice/crio-52fcb1098322c99f5264b3f8325ea9d34f06807725c1ee89f3f513385d64da11 WatchSource:0}: Error finding container 52fcb1098322c99f5264b3f8325ea9d34f06807725c1ee89f3f513385d64da11: Status 404 returned error can't find the container with id 52fcb1098322c99f5264b3f8325ea9d34f06807725c1ee89f3f513385d64da11
Jul 30 01:20:03.911267 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.911236    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:03.911404 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.911312    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:03.911404 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:03.911355    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:04.133267 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:04.133164    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerStarted","Data":"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"}
Jul 30 01:20:04.133267 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:04.133233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerStarted","Data":"52fcb1098322c99f5264b3f8325ea9d34f06807725c1ee89f3f513385d64da11"}
Jul 30 01:20:04.154473 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:04.154393    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podStartSLOduration=1.154369988 podStartE2EDuration="1.154369988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:20:04.152178937 +0000 UTC m=+2191.343127269" watchObservedRunningTime="2026-07-30 01:20:04.154369988 +0000 UTC m=+2191.345318320"
Jul 30 01:20:04.748647 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:04.748608    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:04.753662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:04.753632    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:05.138820 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:05.138725    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:05.139748 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:05.139728    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:14.152174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:14.152143    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:20:21.179976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:21.179934    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 01:20:21.180388 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:21.180047    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 01:20:23.226105 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.226070    2584 generic.go:356] "Generic (PLEG): container finished" podID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerID="9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc" exitCode=0
Jul 30 01:20:23.226560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.226138    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerDied","Data":"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"}
Jul 30 01:20:23.226560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.226512    2584 scope.go:117] "RemoveContainer" containerID="9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"
Jul 30 01:20:23.227544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.227519    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:23.227605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.227586    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:23.227654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:23.227622    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:20:24.234179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:24.234143    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerStarted","Data":"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"}
Jul 30 01:20:24.748009 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:24.747963    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:24.748256 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:24.748233    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 30 01:20:24.748313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:24.748281    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 30 01:20:25.748096 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:25.748054    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 30 01:20:25.748587 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:25.748123    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 30 01:20:26.748404 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:26.748371    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 30 01:20:26.748772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:26.748425    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 30 01:20:27.748058 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:27.748019    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 30 01:20:27.748241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:27.748080    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 30 01:20:28.751675 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:28.751635    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:28.752073 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:28.751900    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:28.752516 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:28.752499    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:20:38.662353 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.662317    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"]
Jul 30 01:20:38.662735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.662653    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:38.662735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.662666    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5f18a7-f167-40cc-91d8-32e5c7c4ae93" containerName="istio-proxy"
Jul 30 01:20:38.665443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.665425    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:38.668418 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.668386    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:20:38.668745 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.668730    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:20:38.750899 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.750866    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"]
Jul 30 01:20:38.772230 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.772197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k\" (UID: \"d373e4b5-0c76-47b5-869d-8c9f096fafc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:38.873132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.873095    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k\" (UID: \"d373e4b5-0c76-47b5-869d-8c9f096fafc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:38.873762 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.873744    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k\" (UID: \"d373e4b5-0c76-47b5-869d-8c9f096fafc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:38.975886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:38.975788    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:39.107394 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:39.107365    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"]
Jul 30 01:20:39.109546 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:20:39.109512    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd373e4b5_0c76_47b5_869d_8c9f096fafc7.slice/crio-df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357 WatchSource:0}: Error finding container df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357: Status 404 returned error can't find the container with id df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357
Jul 30 01:20:39.307739 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:39.307707    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k" event={"ID":"d373e4b5-0c76-47b5-869d-8c9f096fafc7","Type":"ContainerStarted","Data":"df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357"}
Jul 30 01:20:40.314041 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:40.314001    2584 generic.go:356] "Generic (PLEG): container finished" podID="d373e4b5-0c76-47b5-869d-8c9f096fafc7" containerID="a75dc94b20fd11aea9c2889e127e562eb74fe8aecbc2f177be167231a64e1a22" exitCode=0
Jul 30 01:20:40.314421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:40.314071    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k" event={"ID":"d373e4b5-0c76-47b5-869d-8c9f096fafc7","Type":"ContainerDied","Data":"a75dc94b20fd11aea9c2889e127e562eb74fe8aecbc2f177be167231a64e1a22"}
Jul 30 01:20:41.444691 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:41.444662    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:41.495666 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:41.495632    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca\") pod \"d373e4b5-0c76-47b5-869d-8c9f096fafc7\" (UID: \"d373e4b5-0c76-47b5-869d-8c9f096fafc7\") "
Jul 30 01:20:41.496040 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:41.496016    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d373e4b5-0c76-47b5-869d-8c9f096fafc7" (UID: "d373e4b5-0c76-47b5-869d-8c9f096fafc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:20:41.597258 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:41.597169    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d373e4b5-0c76-47b5-869d-8c9f096fafc7-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:20:42.324573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:42.324545    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"
Jul 30 01:20:42.324746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:42.324584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k" event={"ID":"d373e4b5-0c76-47b5-869d-8c9f096fafc7","Type":"ContainerDied","Data":"df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357"}
Jul 30 01:20:42.324746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:42.324633    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df90b377c4cddfa41f81401b6c9bd9ed993ae27c0d86c424fe87361c90638357"
Jul 30 01:20:49.252684 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:49.252647    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"]
Jul 30 01:20:49.257609 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:49.257584    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14e4ae87-qfv9k"]
Jul 30 01:20:49.386556 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:49.386518    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d373e4b5-0c76-47b5-869d-8c9f096fafc7" path="/var/lib/kubelet/pods/d373e4b5-0c76-47b5-869d-8c9f096fafc7/volumes"
Jul 30 01:20:51.941197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:51.941162    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:20:55.663274 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:55.663183    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:20:55.663711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:55.663435    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" containerID="cri-o://6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2" gracePeriod=30
Jul 30 01:20:58.754714 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:58.754678    2584 patch_prober.go:28] interesting pod/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 01:20:58.755173 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:20:58.754741    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 01:21:00.918960 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:00.918931    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:21:01.072594 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072562    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.072863 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072615    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.072863 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072635    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4qxb\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.072863 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072664    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.072863 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072690    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.072863 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072707    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.073150 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.072907    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.073150 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073082    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:01.073150 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073098    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:01.073300 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073163    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:21:01.073300 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073179    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:01.073300 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073276    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.073454 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073314    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data\") pod \"fd14620f-2965-4a45-bc61-3ae50ac95dff\" (UID: \"fd14620f-2965-4a45-bc61-3ae50ac95dff\") "
Jul 30 01:21:01.073658 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073637    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.073731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073665    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd14620f-2965-4a45-bc61-3ae50ac95dff-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.073731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073680    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.073731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.073694    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.074083 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.074062    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:01.074943 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.074913    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:01.075047 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.074962    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:21:01.075184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.075165    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb" (OuterVolumeSpecName: "kube-api-access-f4qxb") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "kube-api-access-f4qxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:21:01.075427 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.075409    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd14620f-2965-4a45-bc61-3ae50ac95dff" (UID: "fd14620f-2965-4a45-bc61-3ae50ac95dff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:21:01.174954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.174906    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.174954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.174948    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.174954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.174962    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.175182 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.174973    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd14620f-2965-4a45-bc61-3ae50ac95dff-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.175182 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.174985    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4qxb\" (UniqueName: \"kubernetes.io/projected/fd14620f-2965-4a45-bc61-3ae50ac95dff-kube-api-access-f4qxb\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:21:01.411990 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.411956    2584 generic.go:356] "Generic (PLEG): container finished" podID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerID="6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2" exitCode=0
Jul 30 01:21:01.412165 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.411998    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerDied","Data":"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"}
Jul 30 01:21:01.412165 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.412014    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"
Jul 30 01:21:01.412165 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.412046    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk" event={"ID":"fd14620f-2965-4a45-bc61-3ae50ac95dff","Type":"ContainerDied","Data":"52fcb1098322c99f5264b3f8325ea9d34f06807725c1ee89f3f513385d64da11"}
Jul 30 01:21:01.412165 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.412069    2584 scope.go:117] "RemoveContainer" containerID="6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"
Jul 30 01:21:01.423856 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.423830    2584 scope.go:117] "RemoveContainer" containerID="9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"
Jul 30 01:21:01.432581 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.432489    2584 scope.go:117] "RemoveContainer" containerID="6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"
Jul 30 01:21:01.433319 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:21:01.433292    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2\": container with ID starting with 6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2 not found: ID does not exist" containerID="6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"
Jul 30 01:21:01.433434 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.433328    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2"} err="failed to get container status \"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2\": rpc error: code = NotFound desc = could not find container \"6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2\": container with ID starting with 6f3922e391659b7d49a6402c63e608311beb499577f23af7e901ca372d9907a2 not found: ID does not exist"
Jul 30 01:21:01.433434 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.433353    2584 scope.go:117] "RemoveContainer" containerID="9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"
Jul 30 01:21:01.433656 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:21:01.433634    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc\": container with ID starting with 9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc not found: ID does not exist" containerID="9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"
Jul 30 01:21:01.433718 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.433665    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc"} err="failed to get container status \"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc\": rpc error: code = NotFound desc = could not find container \"9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc\": container with ID starting with 9ef866ea96de4f518b118647bb258ca0a1ca24ca14833d8a60aa5e0e81b548fc not found: ID does not exist"
Jul 30 01:21:01.434657 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.434636    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:21:01.443828 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:01.440930    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-14e4ae87-openshift-default-75668d99bd-q5lzk"]
Jul 30 01:21:03.385938 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:03.385901    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" path="/var/lib/kubelet/pods/fd14620f-2965-4a45-bc61-3ae50ac95dff/volumes"
Jul 30 01:21:31.046568 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:21:31.046534    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:22:33.465889 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:22:33.463989    2584 scope.go:117] "RemoveContainer" containerID="e40c43b402ecd700cd23260a57f668377f6f7ce33010830f5635ac01d8224f5f"
Jul 30 01:23:10.852198 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852157    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852497    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d373e4b5-0c76-47b5-869d-8c9f096fafc7" containerName="revoke-keys"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852511    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d373e4b5-0c76-47b5-869d-8c9f096fafc7" containerName="revoke-keys"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852526    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852532    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852547    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852553    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852617    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852626    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd14620f-2965-4a45-bc61-3ae50ac95dff" containerName="istio-proxy"
Jul 30 01:23:10.852662 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.852633    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d373e4b5-0c76-47b5-869d-8c9f096fafc7" containerName="revoke-keys"
Jul 30 01:23:10.855555 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.855540    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.858290 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.858259    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:23:10.858431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.858346    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-671ba06b-openshift-default-dockercfg-zhbmz\""
Jul 30 01:23:10.867940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.867909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:23:10.914322 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914292    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914398    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914428    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps8mx\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914616 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914616 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914509    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914616 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914615    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:10.914711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:10.914641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016088 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016103    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps8mx\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016446 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016357    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016525 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016500    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016600 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016610    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016776 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016875 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016854    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.016937 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.016893    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.017055 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.017035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.018722 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.018690    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.018872 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.018737    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.024060 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.024039    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.024147 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.024112    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps8mx\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx\") pod \"e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") " pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.166401 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.166318    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:11.309232 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.309208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:23:11.311186 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:23:11.311159    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f173808_1512_4bde_81ef_390adccea9ba.slice/crio-a372237f50781a6196576326e015704db53a6ef1df4f45f66a5df2d264cfdf48 WatchSource:0}: Error finding container a372237f50781a6196576326e015704db53a6ef1df4f45f66a5df2d264cfdf48: Status 404 returned error can't find the container with id a372237f50781a6196576326e015704db53a6ef1df4f45f66a5df2d264cfdf48
Jul 30 01:23:11.313886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.313849    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:11.313983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.313915    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:11.313983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:11.313946    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:12.012524 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:12.012481    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerStarted","Data":"471143d56ff294fb89a93c32ae72f03295d5916940b002a44fae069d60e5cc5f"}
Jul 30 01:23:12.013039 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:12.012538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerStarted","Data":"a372237f50781a6196576326e015704db53a6ef1df4f45f66a5df2d264cfdf48"}
Jul 30 01:23:12.033150 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:12.033102    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podStartSLOduration=2.03308878 podStartE2EDuration="2.03308878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:23:12.0316201 +0000 UTC m=+2379.222568423" watchObservedRunningTime="2026-07-30 01:23:12.03308878 +0000 UTC m=+2379.224037111"
Jul 30 01:23:12.167180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:12.167138    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:12.171773 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:12.171750    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:13.017216 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:13.017185    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:13.018234 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:13.018212    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:13.970598 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:13.970565    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:23:26.238279 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:26.238237    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:23:29.060995 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:29.060952    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 01:23:29.061459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:29.061023    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 01:23:31.106403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.106368    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f173808-1512-4bde-81ef-390adccea9ba" containerID="471143d56ff294fb89a93c32ae72f03295d5916940b002a44fae069d60e5cc5f" exitCode=0
Jul 30 01:23:31.106848 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.106440    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerDied","Data":"471143d56ff294fb89a93c32ae72f03295d5916940b002a44fae069d60e5cc5f"}
Jul 30 01:23:31.106900 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.106859    2584 scope.go:117] "RemoveContainer" containerID="471143d56ff294fb89a93c32ae72f03295d5916940b002a44fae069d60e5cc5f"
Jul 30 01:23:31.107865 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.107839    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:31.107939 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.107921    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:31.107986 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:31.107972    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:23:32.113773 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:32.113737    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerStarted","Data":"6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820"}
Jul 30 01:23:32.166658 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:32.166617    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:32.166928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:32.166904    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 01:23:32.167023 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:32.166965    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 01:23:33.166882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:33.166850    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 01:23:33.167256 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:33.166898    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 01:23:33.488835 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:33.488788    2584 scope.go:117] "RemoveContainer" containerID="11df90e156046ef96a0dcd9cc9b71ce197f9b013dd7b5aa65e5454eff143f1b5"
Jul 30 01:23:33.496864 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:33.496842    2584 scope.go:117] "RemoveContainer" containerID="1f30b824b85477d888937b76710e778c93c05356028f8f742423ac41e79241fc"
Jul 30 01:23:34.167302 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:34.167265    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 01:23:34.167698 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:34.167323    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 01:23:35.167552 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:35.167518    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 01:23:35.167965 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:35.167582    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 01:23:36.166691 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:36.166639    2584 patch_prober.go:28] interesting pod/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 30 01:23:36.166876 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:36.166728    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 30 01:23:37.170233 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:37.170193    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:37.170656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:37.170472    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:37.171150 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:37.171132    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:23:52.539287 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.539253    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:23:52.542671 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.542653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.545902 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.545871    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-671ba06b\"/\"default-dockercfg-xddlh\""
Jul 30 01:23:52.545902 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.545883    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-671ba06b\"/\"kube-root-ca.crt\""
Jul 30 01:23:52.546088 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.546007    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-671ba06b\"/\"openshift-service-ca.crt\""
Jul 30 01:23:52.546149 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.546120    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-671ba06b\"/\"e2e-lifecycle-model-671ba06b-kserve-self-signed-certs\""
Jul 30 01:23:52.552097 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.552078    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:23:52.603106 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.603241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603121    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.603289 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603237    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.603289 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8rl\" (UniqueName: \"kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.603358 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603306    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.603395 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.603380    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704250 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704271    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8rl\" (UniqueName: \"kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704336    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704380    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704617 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704694 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704668    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704704    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.704854 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.704830    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.706605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.706575    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.706766 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.706749    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.712383 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.712360    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8rl\" (UniqueName: \"kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl\") pod \"e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") " pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.852245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.852165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:23:52.977288 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:52.977261    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:23:52.978947 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:23:52.978919    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6102080a_b147_4b9e_ac55_c275551a496d.slice/crio-7b765289ac7daa9e9691a734590ae44fe1abe4c726d2fc4800daf4d53ffb6667 WatchSource:0}: Error finding container 7b765289ac7daa9e9691a734590ae44fe1abe4c726d2fc4800daf4d53ffb6667: Status 404 returned error can't find the container with id 7b765289ac7daa9e9691a734590ae44fe1abe4c726d2fc4800daf4d53ffb6667
Jul 30 01:23:53.215170 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:53.215081    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" event={"ID":"6102080a-b147-4b9e-ac55-c275551a496d","Type":"ContainerStarted","Data":"cdd3f75c07efcca5496b86ba02f071785488f01009f9d0f9d33b3d7a1417d877"}
Jul 30 01:23:53.215170 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:53.215132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" event={"ID":"6102080a-b147-4b9e-ac55-c275551a496d","Type":"ContainerStarted","Data":"7b765289ac7daa9e9691a734590ae44fe1abe4c726d2fc4800daf4d53ffb6667"}
Jul 30 01:23:53.232090 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:53.232042    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" podStartSLOduration=1.23202668 podStartE2EDuration="1.23202668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:23:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:23:53.231337258 +0000 UTC m=+2420.422285590" watchObservedRunningTime="2026-07-30 01:23:53.23202668 +0000 UTC m=+2420.422975010"
Jul 30 01:23:55.839121 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:23:55.839043    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:24:02.852918 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:02.852876    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:24:02.852918 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:02.852920    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:24:02.865401 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:02.865380    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:24:03.276071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:03.276043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:24:10.847365 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:10.847325    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:24:33.530551 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:33.530512    2584 scope.go:117] "RemoveContainer" containerID="403e973b39b3b3a5c124f590ad1c8dbd5fd295df767d557093eb3e4f5c044d52"
Jul 30 01:24:37.445692 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:37.445661    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:24:47.302844 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:47.302806    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:24:47.966173 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:24:47.966137    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:25:02.326605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.326559    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" podUID="6102080a-b147-4b9e-ac55-c275551a496d" containerName="main" containerID="cri-o://cdd3f75c07efcca5496b86ba02f071785488f01009f9d0f9d33b3d7a1417d877" gracePeriod=45
Jul 30 01:25:02.545855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.545819    2584 generic.go:356] "Generic (PLEG): container finished" podID="6102080a-b147-4b9e-ac55-c275551a496d" containerID="cdd3f75c07efcca5496b86ba02f071785488f01009f9d0f9d33b3d7a1417d877" exitCode=0
Jul 30 01:25:02.545855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.545837    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" event={"ID":"6102080a-b147-4b9e-ac55-c275551a496d","Type":"ContainerDied","Data":"cdd3f75c07efcca5496b86ba02f071785488f01009f9d0f9d33b3d7a1417d877"}
Jul 30 01:25:02.569600 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.569573    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:25:02.640071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.639983    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640021    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f8rl\" (UniqueName: \"kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640046    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640074    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640148    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640186    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm\") pod \"6102080a-b147-4b9e-ac55-c275551a496d\" (UID: \"6102080a-b147-4b9e-ac55-c275551a496d\") "
Jul 30 01:25:02.640405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640291    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home" (OuterVolumeSpecName: "home") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:02.640405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640319    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache" (OuterVolumeSpecName: "model-cache") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:02.640603 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640436    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:02.640603 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640472    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:02.640603 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.640483    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:02.642307 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.642284    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:25:02.642408 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.642312    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm" (OuterVolumeSpecName: "dshm") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:02.642408 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.642387    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl" (OuterVolumeSpecName: "kube-api-access-4f8rl") pod "6102080a-b147-4b9e-ac55-c275551a496d" (UID: "6102080a-b147-4b9e-ac55-c275551a496d"). InnerVolumeSpecName "kube-api-access-4f8rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:25:02.741276 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.741236    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:02.741276 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.741268    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6102080a-b147-4b9e-ac55-c275551a496d-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:02.741276 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.741279    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f8rl\" (UniqueName: \"kubernetes.io/projected/6102080a-b147-4b9e-ac55-c275551a496d-kube-api-access-4f8rl\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:02.741500 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:02.741291    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6102080a-b147-4b9e-ac55-c275551a496d-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:03.551911 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:03.551880    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"
Jul 30 01:25:03.552374 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:03.551879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf" event={"ID":"6102080a-b147-4b9e-ac55-c275551a496d","Type":"ContainerDied","Data":"7b765289ac7daa9e9691a734590ae44fe1abe4c726d2fc4800daf4d53ffb6667"}
Jul 30 01:25:03.552374 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:03.552006    2584 scope.go:117] "RemoveContainer" containerID="cdd3f75c07efcca5496b86ba02f071785488f01009f9d0f9d33b3d7a1417d877"
Jul 30 01:25:03.567604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:03.567579    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:25:03.573302 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:03.573277    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-671ba06b/e2e-lifecycle-model-671ba06b-kserve-7458789c8c-xjtdf"]
Jul 30 01:25:05.385022 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:05.384990    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6102080a-b147-4b9e-ac55-c275551a496d" path="/var/lib/kubelet/pods/6102080a-b147-4b9e-ac55-c275551a496d/volumes"
Jul 30 01:25:44.599829 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:44.599780    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:25:44.600315 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:44.599989    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy" containerID="cri-o://6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820" gracePeriod=30
Jul 30 01:25:47.245827 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:47.245749    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:25:49.637538 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:25:49.637512    2584 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f173808_1512_4bde_81ef_390adccea9ba.slice/crio-6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820.scope\": RecentStats: unable to find data in memory cache]"
Jul 30 01:25:49.762362 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.762330    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f173808-1512-4bde-81ef-390adccea9ba" containerID="6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820" exitCode=0
Jul 30 01:25:49.762546 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.762403    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerDied","Data":"6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820"}
Jul 30 01:25:49.762546 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.762466    2584 scope.go:117] "RemoveContainer" containerID="471143d56ff294fb89a93c32ae72f03295d5916940b002a44fae069d60e5cc5f"
Jul 30 01:25:49.846316 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.846286    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:25:49.859115 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859055    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859189    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859243    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859399 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859280    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859399 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859336    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859399 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859384    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859427    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps8mx\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859480    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859513    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket\") pod \"6f173808-1512-4bde-81ef-390adccea9ba\" (UID: \"6f173808-1512-4bde-81ef-390adccea9ba\") "
Jul 30 01:25:49.859548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859512    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:25:49.859752 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859658    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:49.859896 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859839    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f173808-1512-4bde-81ef-390adccea9ba-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.859974 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859902    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.859974 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.859908    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:49.860085 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.860021    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:49.860255 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.860211    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:49.861620 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.861589    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:25:49.861733 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.861627    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:25:49.861999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.861973    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:25:49.861999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.861980    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx" (OuterVolumeSpecName: "kube-api-access-ps8mx") pod "6f173808-1512-4bde-81ef-390adccea9ba" (UID: "6f173808-1512-4bde-81ef-390adccea9ba"). InnerVolumeSpecName "kube-api-access-ps8mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:25:49.960681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960652    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f173808-1512-4bde-81ef-390adccea9ba-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960679    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960688    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960936 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960696    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960936 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960704    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps8mx\" (UniqueName: \"kubernetes.io/projected/6f173808-1512-4bde-81ef-390adccea9ba-kube-api-access-ps8mx\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960936 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960713    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:49.960936 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:49.960721    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f173808-1512-4bde-81ef-390adccea9ba-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:25:50.768584 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:50.768557    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"
Jul 30 01:25:50.769024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:50.768552    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm" event={"ID":"6f173808-1512-4bde-81ef-390adccea9ba","Type":"ContainerDied","Data":"a372237f50781a6196576326e015704db53a6ef1df4f45f66a5df2d264cfdf48"}
Jul 30 01:25:50.769024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:50.768684    2584 scope.go:117] "RemoveContainer" containerID="6e3fd8ebbc02d840fb50f90b89f78aae16404a94a2ef17825db5659afb4fb820"
Jul 30 01:25:50.790154 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:50.790128    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:25:50.793756 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:50.793732    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-671ba06b-openshift-default-566ff47b-k6jlm"]
Jul 30 01:25:51.385116 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:51.385082    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f173808-1512-4bde-81ef-390adccea9ba" path="/var/lib/kubelet/pods/6f173808-1512-4bde-81ef-390adccea9ba/volumes"
Jul 30 01:25:59.152097 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:25:59.152063    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:26:09.840204 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:09.840159    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:26:49.261441 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.261404    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:26:49.262760 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.262736    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:26:49.262939 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.262927    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:26:49.263027 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.263018    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6102080a-b147-4b9e-ac55-c275551a496d" containerName="main"
Jul 30 01:26:49.263097 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.263089    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6102080a-b147-4b9e-ac55-c275551a496d" containerName="main"
Jul 30 01:26:49.263357 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.263345    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6102080a-b147-4b9e-ac55-c275551a496d" containerName="main"
Jul 30 01:26:49.263443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.263435    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:26:49.263523 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.263514    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:26:49.267466 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.267443    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.270040 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.270020    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:26:49.270040 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.270031    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6ce027-openshift-default-dockercfg-splqs\""
Jul 30 01:26:49.273885 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.273862    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:26:49.375813 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.375770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.375824    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.375920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.375976    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.375997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.376027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.376061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.376082    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thn5n\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.376174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.376135    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477201    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477242    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thn5n\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477518    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477788 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.477963 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477902    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.478027 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.477983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.478230 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.478206    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.478401 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.478376    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.478492 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.478469    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.478612 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.478595    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.480119 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.480087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.480222 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.480175    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.485073 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.485054    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.485402 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.485379    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thn5n\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n\") pod \"e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") " pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.579055 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.578969    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:49.718248 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.718213    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:26:49.720344 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:26:49.720319    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac1b8dc_8647_43c6_a1d0_8c08226ca1a7.slice/crio-9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160 WatchSource:0}: Error finding container 9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160: Status 404 returned error can't find the container with id 9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160
Jul 30 01:26:49.722840 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.722809    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:26:49.722942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.722887    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:26:49.722942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:49.722930    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:26:50.039864 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:50.039832    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerStarted","Data":"986de9713843c0bf07974703166a4cb75494d05fb1b10893c0c58a7267973f99"}
Jul 30 01:26:50.040033 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:50.039887    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerStarted","Data":"9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160"}
Jul 30 01:26:50.085543 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:50.085496    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podStartSLOduration=1.085481179 podStartE2EDuration="1.085481179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:26:50.083829505 +0000 UTC m=+2597.274777836" watchObservedRunningTime="2026-07-30 01:26:50.085481179 +0000 UTC m=+2597.276429509"
Jul 30 01:26:50.579573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:50.579529    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:50.584160 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:50.584135    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:51.044774 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:51.044745    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:51.045763 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:51.045745    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:26:59.337019 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:26:59.336942    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:27:07.085976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:07.085938    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 01:27:07.087303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:07.086001    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 01:27:08.119922 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.119889    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerID="986de9713843c0bf07974703166a4cb75494d05fb1b10893c0c58a7267973f99" exitCode=0
Jul 30 01:27:08.120407 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.119970    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerDied","Data":"986de9713843c0bf07974703166a4cb75494d05fb1b10893c0c58a7267973f99"}
Jul 30 01:27:08.120478 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.120465    2584 scope.go:117] "RemoveContainer" containerID="986de9713843c0bf07974703166a4cb75494d05fb1b10893c0c58a7267973f99"
Jul 30 01:27:08.121468 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.121441    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:27:08.121564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.121521    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:27:08.121627 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:08.121579    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:27:09.129533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:09.129485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerStarted","Data":"350ce63ddd154a20e9a14bd2116b62d65d5422720c46e27145f4f99eacaf9d28"}
Jul 30 01:27:09.579470 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:09.579433    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:27:09.579697 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:09.579674    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 30 01:27:09.579755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:09.579736    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 30 01:27:10.579820 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:10.579768    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 30 01:27:10.580205 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:10.579858    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 30 01:27:11.580031 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:11.579989    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 30 01:27:11.580500 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:11.580054    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 30 01:27:12.579630 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:12.579593    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 30 01:27:12.579829 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:12.579645    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 30 01:27:13.583303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:13.583271    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:27:13.583670 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:13.583538    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:27:13.584329 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:13.584297    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:27:28.646704 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:28.646665    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:27:33.569763 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:33.569731    2584 scope.go:117] "RemoveContainer" containerID="a75dc94b20fd11aea9c2889e127e562eb74fe8aecbc2f177be167231a64e1a22"
Jul 30 01:27:48.944932 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.944896    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:27:48.945307 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.945233    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:27:48.945307 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.945244    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f173808-1512-4bde-81ef-390adccea9ba" containerName="istio-proxy"
Jul 30 01:27:48.947032 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.947017    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:48.950461 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.950437    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6ce027\"/\"test-model-6ce027-kserve-self-signed-certs\""
Jul 30 01:27:48.950600 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.950437    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6ce027\"/\"openshift-service-ca.crt\""
Jul 30 01:27:48.950674 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.950597    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6ce027\"/\"default-dockercfg-m8xzc\""
Jul 30 01:27:48.950674 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.950443    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6ce027\"/\"kube-root-ca.crt\""
Jul 30 01:27:48.956539 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:48.956518    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:27:49.005477 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005443    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.005631 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005493    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.005631 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005518    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhq5\" (UniqueName: \"kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.005631 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005602    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.005818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.005818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.005727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106582 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106761 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhq5\" (UniqueName: \"kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106761 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106654    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106761 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106726    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106933 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106783    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.106933 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.106855    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.107216 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.107188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.107629 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.107230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.107741 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.107572    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.109378 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.109355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.109488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.109431    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.116349 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.116320    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhq5\" (UniqueName: \"kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5\") pod \"test-model-6ce027-kserve-696679fddc-k2hz8\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") " pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.257839 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.257772    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:49.407251 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:49.407227    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:27:49.409659 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:27:49.409634    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79628d68_fe83_4f8d_b6fc_8bbc4d29311d.slice/crio-86228eec1c19346298f13ad3890cbd3bf63a8f626939edfb9f2ebf5522fe9aea WatchSource:0}: Error finding container 86228eec1c19346298f13ad3890cbd3bf63a8f626939edfb9f2ebf5522fe9aea: Status 404 returned error can't find the container with id 86228eec1c19346298f13ad3890cbd3bf63a8f626939edfb9f2ebf5522fe9aea
Jul 30 01:27:50.317850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:50.317809    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" event={"ID":"79628d68-fe83-4f8d-b6fc-8bbc4d29311d","Type":"ContainerStarted","Data":"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"}
Jul 30 01:27:50.317850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:50.317859    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" event={"ID":"79628d68-fe83-4f8d-b6fc-8bbc4d29311d","Type":"ContainerStarted","Data":"86228eec1c19346298f13ad3890cbd3bf63a8f626939edfb9f2ebf5522fe9aea"}
Jul 30 01:27:50.333258 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:50.333214    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" podStartSLOduration=2.333199115 podStartE2EDuration="2.333199115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:27:50.33307101 +0000 UTC m=+2657.524019342" watchObservedRunningTime="2026-07-30 01:27:50.333199115 +0000 UTC m=+2657.524147424"
Jul 30 01:27:52.542078 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:52.542045    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:27:59.258425 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:59.258390    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:59.258425 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:59.258436    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:59.270757 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:59.270735    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:27:59.371810 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:27:59.371762    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:28:04.139108 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:04.139071    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:28:39.455160 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.455081    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:28:39.457537 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.457519    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.460072 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.460050    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b2ddd3\"/\"kube-root-ca.crt\""
Jul 30 01:28:39.461060 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.461038    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b2ddd3\"/\"test-model-b2ddd3-kserve-self-signed-certs\""
Jul 30 01:28:39.461175 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.461071    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b2ddd3\"/\"default-dockercfg-bfz8t\""
Jul 30 01:28:39.461175 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.461083    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b2ddd3\"/\"openshift-service-ca.crt\""
Jul 30 01:28:39.466573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.466552    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:28:39.558546 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558509    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.558710 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.558710 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558649    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.558710 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558693    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.558869 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.558869 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.558759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsw7r\" (UniqueName: \"kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659504 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659678 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659678 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659560    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659678 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659592    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659678 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659644    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsw7r\" (UniqueName: \"kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.659928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.659683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.660028 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.660000    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.660098 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.660034    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.660098 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.660045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.662024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.662002    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.662278 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.662256    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.667898 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.667875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsw7r\" (UniqueName: \"kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r\") pod \"test-model-b2ddd3-kserve-7cdfd85f79-x8mgg\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") " pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.767961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.767927    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:39.895393 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:39.895366    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:28:39.898293 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:28:39.898261    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5adf429b_92a5_4fad_86b1_dac1dd1bb661.slice/crio-a0413d0e3c153dab3e3df9311314385ab24e8f51ba1b38b5245b0ccfd088ad5a WatchSource:0}: Error finding container a0413d0e3c153dab3e3df9311314385ab24e8f51ba1b38b5245b0ccfd088ad5a: Status 404 returned error can't find the container with id a0413d0e3c153dab3e3df9311314385ab24e8f51ba1b38b5245b0ccfd088ad5a
Jul 30 01:28:40.552680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:40.552643    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" event={"ID":"5adf429b-92a5-4fad-86b1-dac1dd1bb661","Type":"ContainerStarted","Data":"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"}
Jul 30 01:28:40.552680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:40.552687    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" event={"ID":"5adf429b-92a5-4fad-86b1-dac1dd1bb661","Type":"ContainerStarted","Data":"a0413d0e3c153dab3e3df9311314385ab24e8f51ba1b38b5245b0ccfd088ad5a"}
Jul 30 01:28:40.569534 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:40.569490    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" podStartSLOduration=1.5694743610000002 podStartE2EDuration="1.569474361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:28:40.567955559 +0000 UTC m=+2707.758903889" watchObservedRunningTime="2026-07-30 01:28:40.569474361 +0000 UTC m=+2707.760422691"
Jul 30 01:28:43.742779 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:43.742741    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:28:49.768385 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:49.768352    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:49.768385 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:49.768405    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:49.780850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:49.780826    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:50.615021 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:50.614979    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:28:58.152155 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:28:58.152116    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:29:20.333636 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:20.333600    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:29:21.326855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:21.326818    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:29:25.040977 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:25.040940    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:29:35.353654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.353589    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" podUID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" containerName="main" containerID="cri-o://3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5" gracePeriod=45
Jul 30 01:29:35.607755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.607693    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:29:35.671631 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671601    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.671825 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671644    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.671825 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671666    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.671825 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671686    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhhq5\" (UniqueName: \"kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.671825 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671708    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.672071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671896    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir\") pod \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\" (UID: \"79628d68-fe83-4f8d-b6fc-8bbc4d29311d\") "
Jul 30 01:29:35.672071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671941    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache" (OuterVolumeSpecName: "model-cache") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:35.672071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.671983    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home" (OuterVolumeSpecName: "home") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:35.672200 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.672103    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:35.672369 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.672331    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.672369 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.672357    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.672369 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.672371    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.673947 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.673920    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm" (OuterVolumeSpecName: "dshm") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:35.673947 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.673932    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:29:35.674057 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.674003    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5" (OuterVolumeSpecName: "kube-api-access-jhhq5") pod "79628d68-fe83-4f8d-b6fc-8bbc4d29311d" (UID: "79628d68-fe83-4f8d-b6fc-8bbc4d29311d"). InnerVolumeSpecName "kube-api-access-jhhq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:35.773135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.773103    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.773135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.773130    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhhq5\" (UniqueName: \"kubernetes.io/projected/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-kube-api-access-jhhq5\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.773135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.773139    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79628d68-fe83-4f8d-b6fc-8bbc4d29311d-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:35.803546 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.803511    2584 generic.go:356] "Generic (PLEG): container finished" podID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" containerID="3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5" exitCode=0
Jul 30 01:29:35.803707 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.803553    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" event={"ID":"79628d68-fe83-4f8d-b6fc-8bbc4d29311d","Type":"ContainerDied","Data":"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"}
Jul 30 01:29:35.803707 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.803584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8" event={"ID":"79628d68-fe83-4f8d-b6fc-8bbc4d29311d","Type":"ContainerDied","Data":"86228eec1c19346298f13ad3890cbd3bf63a8f626939edfb9f2ebf5522fe9aea"}
Jul 30 01:29:35.803707 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.803601    2584 scope.go:117] "RemoveContainer" containerID="3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"
Jul 30 01:29:35.803707 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.803558    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"
Jul 30 01:29:35.812350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.812332    2584 scope.go:117] "RemoveContainer" containerID="3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"
Jul 30 01:29:35.812586 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:29:35.812565    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5\": container with ID starting with 3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5 not found: ID does not exist" containerID="3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"
Jul 30 01:29:35.812667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.812594    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5"} err="failed to get container status \"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5\": rpc error: code = NotFound desc = could not find container \"3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5\": container with ID starting with 3e99ed00549f4b2756f3460e9d3b306a4371adb26f315a57198c9650c89f31e5 not found: ID does not exist"
Jul 30 01:29:35.824879 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.824854    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:29:35.828331 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:35.828309    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6ce027/test-model-6ce027-kserve-696679fddc-k2hz8"]
Jul 30 01:29:36.345784 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.345745    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" podUID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" containerName="main" containerID="cri-o://667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc" gracePeriod=45
Jul 30 01:29:36.605669 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.605608    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:29:36.680638 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680608    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsw7r\" (UniqueName: \"kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.680850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680670    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.680850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680689    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.680850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680719    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.680850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680738    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.680850 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680816    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir\") pod \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\" (UID: \"5adf429b-92a5-4fad-86b1-dac1dd1bb661\") "
Jul 30 01:29:36.681130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.680931    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache" (OuterVolumeSpecName: "model-cache") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:36.681130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.681004    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home" (OuterVolumeSpecName: "home") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:36.681130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.681108    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:36.681239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.681222    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.681338 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.681244    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.681338 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.681258    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.683114 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.683088    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm" (OuterVolumeSpecName: "dshm") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:36.683226 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.683161    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r" (OuterVolumeSpecName: "kube-api-access-jsw7r") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "kube-api-access-jsw7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:36.683226 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.683165    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5adf429b-92a5-4fad-86b1-dac1dd1bb661" (UID: "5adf429b-92a5-4fad-86b1-dac1dd1bb661"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:29:36.782260 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.782224    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5adf429b-92a5-4fad-86b1-dac1dd1bb661-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.782260 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.782252    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5adf429b-92a5-4fad-86b1-dac1dd1bb661-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.782260 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.782261    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsw7r\" (UniqueName: \"kubernetes.io/projected/5adf429b-92a5-4fad-86b1-dac1dd1bb661-kube-api-access-jsw7r\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:36.809905 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.809869    2584 generic.go:356] "Generic (PLEG): container finished" podID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" containerID="667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc" exitCode=0
Jul 30 01:29:36.810047 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.809919    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"
Jul 30 01:29:36.810047 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.809957    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" event={"ID":"5adf429b-92a5-4fad-86b1-dac1dd1bb661","Type":"ContainerDied","Data":"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"}
Jul 30 01:29:36.810047 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.809999    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg" event={"ID":"5adf429b-92a5-4fad-86b1-dac1dd1bb661","Type":"ContainerDied","Data":"a0413d0e3c153dab3e3df9311314385ab24e8f51ba1b38b5245b0ccfd088ad5a"}
Jul 30 01:29:36.810047 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.810016    2584 scope.go:117] "RemoveContainer" containerID="667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"
Jul 30 01:29:36.820933 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.820899    2584 scope.go:117] "RemoveContainer" containerID="667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"
Jul 30 01:29:36.821252 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:29:36.821233    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc\": container with ID starting with 667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc not found: ID does not exist" containerID="667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"
Jul 30 01:29:36.821344 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.821258    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc"} err="failed to get container status \"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc\": rpc error: code = NotFound desc = could not find container \"667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc\": container with ID starting with 667a4c24b60b82fcc3b4ea4e9d226b1fb16b09ec943105dba33fa4f806c317dc not found: ID does not exist"
Jul 30 01:29:36.830463 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.830437    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:29:36.836873 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:36.836854    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b2ddd3/test-model-b2ddd3-kserve-7cdfd85f79-x8mgg"]
Jul 30 01:29:37.385857 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:37.385823    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" path="/var/lib/kubelet/pods/5adf429b-92a5-4fad-86b1-dac1dd1bb661/volumes"
Jul 30 01:29:37.386248 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:37.386235    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" path="/var/lib/kubelet/pods/79628d68-fe83-4f8d-b6fc-8bbc4d29311d/volumes"
Jul 30 01:29:38.608517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.608482    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:29:38.609009 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.608731    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" containerID="cri-o://350ce63ddd154a20e9a14bd2116b62d65d5422720c46e27145f4f99eacaf9d28" gracePeriod=30
Jul 30 01:29:38.977140 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977056    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"]
Jul 30 01:29:38.977435 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977422    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" containerName="main"
Jul 30 01:29:38.977482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977436    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" containerName="main"
Jul 30 01:29:38.977482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977447    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" containerName="main"
Jul 30 01:29:38.977482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977452    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" containerName="main"
Jul 30 01:29:38.977576 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977528    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5adf429b-92a5-4fad-86b1-dac1dd1bb661" containerName="main"
Jul 30 01:29:38.977576 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.977536    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="79628d68-fe83-4f8d-b6fc-8bbc4d29311d" containerName="main"
Jul 30 01:29:38.980234 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.980216    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:38.982405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.982381    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:29:38.982511 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.982464    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:29:38.987981 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:38.987960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"]
Jul 30 01:29:39.002570 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.002545    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx\" (UID: \"93baa675-2675-4c7d-ac72-573657920b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:39.103602 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.103568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx\" (UID: \"93baa675-2675-4c7d-ac72-573657920b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:39.104307 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.104279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx\" (UID: \"93baa675-2675-4c7d-ac72-573657920b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:39.291744 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.291697    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:39.423515 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.423491    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"]
Jul 30 01:29:39.425833 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:29:39.425787    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93baa675_2675_4c7d_ac72_573657920b57.slice/crio-ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215 WatchSource:0}: Error finding container ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215: Status 404 returned error can't find the container with id ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215
Jul 30 01:29:39.826841 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.826789    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx" event={"ID":"93baa675-2675-4c7d-ac72-573657920b57","Type":"ContainerStarted","Data":"7c790e6524c3f058970ae4fb25efd7ced8341c576042e324c589b9726f694335"}
Jul 30 01:29:39.827228 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.826864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx" event={"ID":"93baa675-2675-4c7d-ac72-573657920b57","Type":"ContainerStarted","Data":"ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215"}
Jul 30 01:29:39.854714 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:39.854654    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx" podStartSLOduration=1.8546393920000002 podStartE2EDuration="1.854639392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:29:39.852440463 +0000 UTC m=+2767.043388791" watchObservedRunningTime="2026-07-30 01:29:39.854639392 +0000 UTC m=+2767.045587723"
Jul 30 01:29:40.832530 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:40.832494    2584 generic.go:356] "Generic (PLEG): container finished" podID="93baa675-2675-4c7d-ac72-573657920b57" containerID="7c790e6524c3f058970ae4fb25efd7ced8341c576042e324c589b9726f694335" exitCode=0
Jul 30 01:29:40.833016 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:40.832578    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx" event={"ID":"93baa675-2675-4c7d-ac72-573657920b57","Type":"ContainerDied","Data":"7c790e6524c3f058970ae4fb25efd7ced8341c576042e324c589b9726f694335"}
Jul 30 01:29:40.940732 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:40.940698    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:29:41.964651 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:41.964628    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:42.030838 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.030788    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca\") pod \"93baa675-2675-4c7d-ac72-573657920b57\" (UID: \"93baa675-2675-4c7d-ac72-573657920b57\") "
Jul 30 01:29:42.031168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.031144    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93baa675-2675-4c7d-ac72-573657920b57" (UID: "93baa675-2675-4c7d-ac72-573657920b57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:42.131464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.131364    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93baa675-2675-4c7d-ac72-573657920b57-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:42.842685 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.842656    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"
Jul 30 01:29:42.842882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.842688    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx" event={"ID":"93baa675-2675-4c7d-ac72-573657920b57","Type":"ContainerDied","Data":"ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215"}
Jul 30 01:29:42.842882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:42.842736    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad433cfa413b3183d8a55a7068d3520b71b3f07afaf07da2fefc2c53ba3d5215"
Jul 30 01:29:43.586405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.586371    2584 patch_prober.go:28] interesting pod/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 01:29:43.586764 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.586428    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 01:29:43.849056 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.849030    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerID="350ce63ddd154a20e9a14bd2116b62d65d5422720c46e27145f4f99eacaf9d28" exitCode=0
Jul 30 01:29:43.849176 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.849119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerDied","Data":"350ce63ddd154a20e9a14bd2116b62d65d5422720c46e27145f4f99eacaf9d28"}
Jul 30 01:29:43.849220 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.849190    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl" event={"ID":"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7","Type":"ContainerDied","Data":"9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160"}
Jul 30 01:29:43.849259 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.849216    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9220835fa622f8ea64321074c20bb817d67c4be574f972701c91cd5ef0a0e160"
Jul 30 01:29:43.849259 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.849237    2584 scope.go:117] "RemoveContainer" containerID="986de9713843c0bf07974703166a4cb75494d05fb1b10893c0c58a7267973f99"
Jul 30 01:29:43.858970 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.858953    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:29:43.946680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946652    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946685    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946700    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946716    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946732    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946753    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946779    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946841    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.946966 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.946947    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thn5n\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n\") pod \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\" (UID: \"9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7\") "
Jul 30 01:29:43.947341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947059    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:43.947341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947170    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:43.947341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947282    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:43.947341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947291    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:43.947341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947299    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:43.947579 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947459    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:43.947646 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.947619    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:43.949053 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.949030    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:29:43.949263 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.949238    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n" (OuterVolumeSpecName: "kube-api-access-thn5n") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "kube-api-access-thn5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:43.949414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.949400    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:29:43.949466 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:43.949454    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" (UID: "9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:44.048735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048700    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thn5n\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-kube-api-access-thn5n\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.048735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048729    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.048735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048738    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.048735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048748    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.049013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048756    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.049013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048765    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.049013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.048773    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:29:44.855360 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.855331    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"
Jul 30 01:29:44.893579 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.893542    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:29:44.899616 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:44.899589    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6ce027-openshift-default-59dfc7d89b-kckdl"]
Jul 30 01:29:45.386034 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:45.386000    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" path="/var/lib/kubelet/pods/9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7/volumes"
Jul 30 01:29:49.608747 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:49.608712    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"]
Jul 30 01:29:49.613212 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:49.613183    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2ddd3-4pzqx"]
Jul 30 01:29:51.385664 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:51.385630    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93baa675-2675-4c7d-ac72-573657920b57" path="/var/lib/kubelet/pods/93baa675-2675-4c7d-ac72-573657920b57/volumes"
Jul 30 01:29:52.835171 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:52.835139    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:29:56.898216 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898132    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898608    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898628    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898652    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898661    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898701    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93baa675-2675-4c7d-ac72-573657920b57" containerName="revoke-keys"
Jul 30 01:29:56.898755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898710    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="93baa675-2675-4c7d-ac72-573657920b57" containerName="revoke-keys"
Jul 30 01:29:56.899090 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898789    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="93baa675-2675-4c7d-ac72-573657920b57" containerName="revoke-keys"
Jul 30 01:29:56.899090 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898832    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.899090 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.898842    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac1b8dc-8647-43c6-a1d0-8c08226ca1a7" containerName="istio-proxy"
Jul 30 01:29:56.903098 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.903075    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.905481 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.905458    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:29:56.905588 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.905494    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5ca190-openshift-default-dockercfg-bqrk7\""
Jul 30 01:29:56.914006 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.913982    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:29:56.959969 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.959938    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.959977    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.959997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vcsd\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960132    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960347 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960274    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:56.960347 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:56.960310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.060908 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.060873    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.060914    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.060941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vcsd\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.060984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061048    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061277 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061072    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061277 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061277 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061430 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061309    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061430 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061381    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061608 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061488    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061608 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061571    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.061943 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.061920    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.063202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.063182    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.063580 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.063561    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.069120 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.069095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.069208 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.069123    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vcsd\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd\") pod \"e2e-api-a-5ca190-openshift-default-6944747694-zwt8q\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") " pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.214327 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.214240    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:57.346242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.346210    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:29:57.348341 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:29:57.348305    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ae413a_2e7f_47ed_9d1a_172bfdc08f28.slice/crio-7fa0af0f6578408f9652995e57449d74018fdc092c09674745c52fb4960adb94 WatchSource:0}: Error finding container 7fa0af0f6578408f9652995e57449d74018fdc092c09674745c52fb4960adb94: Status 404 returned error can't find the container with id 7fa0af0f6578408f9652995e57449d74018fdc092c09674745c52fb4960adb94
Jul 30 01:29:57.350436 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.350404    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:29:57.350499 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.350475    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:29:57.350538 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.350503    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:29:57.914285 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.914243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerStarted","Data":"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"}
Jul 30 01:29:57.914670 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.914313    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerStarted","Data":"7fa0af0f6578408f9652995e57449d74018fdc092c09674745c52fb4960adb94"}
Jul 30 01:29:57.935176 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:57.935129    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podStartSLOduration=1.935114322 podStartE2EDuration="1.935114322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:29:57.933021255 +0000 UTC m=+2785.123969589" watchObservedRunningTime="2026-07-30 01:29:57.935114322 +0000 UTC m=+2785.126062653"
Jul 30 01:29:58.214913 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:58.214823    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:58.219700 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:58.219671    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:58.919557 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:58.919523    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:29:58.920607 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:29:58.920583    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:00.129239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.129205    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"]
Jul 30 01:30:00.132594 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.132577    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"
Jul 30 01:30:00.135043 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.135020    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:30:00.140445 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.140421    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"]
Jul 30 01:30:00.144025 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.144002    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"
Jul 30 01:30:00.271485 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.271459    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"]
Jul 30 01:30:00.273033 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:30:00.273004    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61533aa_e622_4dfa_bb6f_418a277c7c0f.slice/crio-40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489 WatchSource:0}: Error finding container 40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489: Status 404 returned error can't find the container with id 40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489
Jul 30 01:30:00.931853 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.931723    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns" event={"ID":"f61533aa-e622-4dfa-bb6f-418a277c7c0f","Type":"ContainerStarted","Data":"b7377e498e9e88f0d4d271c9ce80a278a586eb5bc685bc4f4a408b9a40f23f7d"}
Jul 30 01:30:00.931853 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.931788    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns" event={"ID":"f61533aa-e622-4dfa-bb6f-418a277c7c0f","Type":"ContainerStarted","Data":"40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489"}
Jul 30 01:30:00.953510 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:00.953442    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns" podStartSLOduration=0.953421671 podStartE2EDuration="953.421671ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:00.951672705 +0000 UTC m=+2788.142621037" watchObservedRunningTime="2026-07-30 01:30:00.953421671 +0000 UTC m=+2788.144370004"
Jul 30 01:30:01.938113 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:01.938077    2584 generic.go:356] "Generic (PLEG): container finished" podID="f61533aa-e622-4dfa-bb6f-418a277c7c0f" containerID="b7377e498e9e88f0d4d271c9ce80a278a586eb5bc685bc4f4a408b9a40f23f7d" exitCode=0
Jul 30 01:30:01.938573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:01.938171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns" event={"ID":"f61533aa-e622-4dfa-bb6f-418a277c7c0f","Type":"ContainerDied","Data":"b7377e498e9e88f0d4d271c9ce80a278a586eb5bc685bc4f4a408b9a40f23f7d"}
Jul 30 01:30:03.071330 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:03.071298    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"
Jul 30 01:30:03.948900 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:03.948869    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns"
Jul 30 01:30:03.948900 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:03.948895    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756250-dwdns" event={"ID":"f61533aa-e622-4dfa-bb6f-418a277c7c0f","Type":"ContainerDied","Data":"40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489"}
Jul 30 01:30:03.949162 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:03.948945    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40c69876187f1a8dce887fa8fe14bb0045b07f5c578bda42f737ba882bdb1489"
Jul 30 01:30:04.546355 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:04.546313    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:30:12.994859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.994827    2584 generic.go:356] "Generic (PLEG): container finished" podID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerID="a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f" exitCode=0
Jul 30 01:30:12.995312 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.994901    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerDied","Data":"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"}
Jul 30 01:30:12.995312 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.995300    2584 scope.go:117] "RemoveContainer" containerID="a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"
Jul 30 01:30:12.996310 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.996280    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:30:12.996409 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.996375    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:30:12.996479 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:12.996424    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:30:13.920464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:13.920414    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:14.001708 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:14.001672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerStarted","Data":"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"}
Jul 30 01:30:14.216015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:14.215861    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:14.216517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:14.216490    2584 patch_prober.go:28] interesting pod/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 01:30:14.216682 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:14.216657    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 01:30:15.215219 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:15.215171    2584 patch_prober.go:28] interesting pod/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 01:30:15.215772 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:15.215252    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 01:30:16.215242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:16.215198    2584 patch_prober.go:28] interesting pod/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 01:30:16.215816 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:16.215272    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 01:30:17.214725 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:17.214677    2584 patch_prober.go:28] interesting pod/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 01:30:17.214952 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:17.214746    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 01:30:18.218925 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:18.218894    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:18.219339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:18.219312    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:18.219996 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:18.219976    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:30:37.338993 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:37.338952    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:30:56.385840 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.385790    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"]
Jul 30 01:30:56.386313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.386135    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61533aa-e622-4dfa-bb6f-418a277c7c0f" containerName="cleanup"
Jul 30 01:30:56.386313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.386147    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61533aa-e622-4dfa-bb6f-418a277c7c0f" containerName="cleanup"
Jul 30 01:30:56.386313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.386198    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61533aa-e622-4dfa-bb6f-418a277c7c0f" containerName="cleanup"
Jul 30 01:30:56.388389 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.388371    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:56.390830 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.390805    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:30:56.390961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.390839    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:30:56.395418 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.395390    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"]
Jul 30 01:30:56.481529 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.481492    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5ca190-szwgw\" (UID: \"1f1850c3-7273-4434-a42a-820271375c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:56.582994 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.582955    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5ca190-szwgw\" (UID: \"1f1850c3-7273-4434-a42a-820271375c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:56.583599 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.583576    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5ca190-szwgw\" (UID: \"1f1850c3-7273-4434-a42a-820271375c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:56.699618 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.699524    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:56.841923 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:56.841894    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"]
Jul 30 01:30:56.843822 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:30:56.843766    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1850c3_7273_4434_a42a_820271375c5b.slice/crio-d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637 WatchSource:0}: Error finding container d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637: Status 404 returned error can't find the container with id d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637
Jul 30 01:30:57.212160 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:57.212118    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw" event={"ID":"1f1850c3-7273-4434-a42a-820271375c5b","Type":"ContainerStarted","Data":"c9a076d37e835bafa53205fc4a6102427d574c25644175610d50cbae6d094ac7"}
Jul 30 01:30:57.212320 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:57.212214    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw" event={"ID":"1f1850c3-7273-4434-a42a-820271375c5b","Type":"ContainerStarted","Data":"d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637"}
Jul 30 01:30:57.228071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:57.228021    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw" podStartSLOduration=1.22800689 podStartE2EDuration="1.22800689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:57.226354579 +0000 UTC m=+2844.417302913" watchObservedRunningTime="2026-07-30 01:30:57.22800689 +0000 UTC m=+2844.418955220"
Jul 30 01:30:58.218136 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:58.218105    2584 generic.go:356] "Generic (PLEG): container finished" podID="1f1850c3-7273-4434-a42a-820271375c5b" containerID="c9a076d37e835bafa53205fc4a6102427d574c25644175610d50cbae6d094ac7" exitCode=0
Jul 30 01:30:58.218136 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:58.218140    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw" event={"ID":"1f1850c3-7273-4434-a42a-820271375c5b","Type":"ContainerDied","Data":"c9a076d37e835bafa53205fc4a6102427d574c25644175610d50cbae6d094ac7"}
Jul 30 01:30:59.347879 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:59.347851    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:30:59.508173 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:59.508142    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca\") pod \"1f1850c3-7273-4434-a42a-820271375c5b\" (UID: \"1f1850c3-7273-4434-a42a-820271375c5b\") "
Jul 30 01:30:59.508547 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:59.508523    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f1850c3-7273-4434-a42a-820271375c5b" (UID: "1f1850c3-7273-4434-a42a-820271375c5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:30:59.609577 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:30:59.609542    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f1850c3-7273-4434-a42a-820271375c5b-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:00.228678 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:00.228643    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw" event={"ID":"1f1850c3-7273-4434-a42a-820271375c5b","Type":"ContainerDied","Data":"d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637"}
Jul 30 01:31:00.228877 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:00.228698    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d787018e1ed15ccc0d1a904d8aab68d677dc982190c0437fb14beed3c9cef637"
Jul 30 01:31:00.228877 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:00.228653    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"
Jul 30 01:31:12.165094 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:12.165050    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"]
Jul 30 01:31:12.167125 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:12.167102    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5ca190-szwgw"]
Jul 30 01:31:13.385393 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:13.385359    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1850c3-7273-4434-a42a-820271375c5b" path="/var/lib/kubelet/pods/1f1850c3-7273-4434-a42a-820271375c5b/volumes"
Jul 30 01:31:14.145726 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:14.145697    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:31:18.613015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.612984    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:31:18.613510 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.613213    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy" containerID="cri-o://54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c" gracePeriod=30
Jul 30 01:31:18.981749 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.981654    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"]
Jul 30 01:31:18.982064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.982047    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1850c3-7273-4434-a42a-820271375c5b" containerName="revoke-keys"
Jul 30 01:31:18.982132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.982066    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1850c3-7273-4434-a42a-820271375c5b" containerName="revoke-keys"
Jul 30 01:31:18.982132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.982121    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1850c3-7273-4434-a42a-820271375c5b" containerName="revoke-keys"
Jul 30 01:31:18.984044 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.984025    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:18.988415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.988393    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:31:18.988415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.988406    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:31:18.995144 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:18.995120    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"]
Jul 30 01:31:19.053740 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:19.053701    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72\" (UID: \"3a81a930-1e79-4c09-89d7-abd396442ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:19.155268 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:19.155224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72\" (UID: \"3a81a930-1e79-4c09-89d7-abd396442ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:19.155908 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:19.155884    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72\" (UID: \"3a81a930-1e79-4c09-89d7-abd396442ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:19.295288 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:19.295249    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:19.428824 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:19.428779    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"]
Jul 30 01:31:19.431109 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:31:19.431078    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a81a930_1e79_4c09_89d7_abd396442ffc.slice/crio-cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99 WatchSource:0}: Error finding container cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99: Status 404 returned error can't find the container with id cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99
Jul 30 01:31:20.320960 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:20.320923    2584 generic.go:356] "Generic (PLEG): container finished" podID="3a81a930-1e79-4c09-89d7-abd396442ffc" containerID="bbba76e1ce5a0b3d5614d5e4da9955f7ac6c844d4ef492bd073707e9a2088dea" exitCode=0
Jul 30 01:31:20.321355 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:20.320976    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72" event={"ID":"3a81a930-1e79-4c09-89d7-abd396442ffc","Type":"ContainerDied","Data":"bbba76e1ce5a0b3d5614d5e4da9955f7ac6c844d4ef492bd073707e9a2088dea"}
Jul 30 01:31:20.321355 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:20.321025    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72" event={"ID":"3a81a930-1e79-4c09-89d7-abd396442ffc","Type":"ContainerStarted","Data":"cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99"}
Jul 30 01:31:21.455139 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:21.455114    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:21.471728 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:21.471691    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca\") pod \"3a81a930-1e79-4c09-89d7-abd396442ffc\" (UID: \"3a81a930-1e79-4c09-89d7-abd396442ffc\") "
Jul 30 01:31:21.472297 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:21.472255    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a81a930-1e79-4c09-89d7-abd396442ffc" (UID: "3a81a930-1e79-4c09-89d7-abd396442ffc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:21.572548 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:21.572519    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a81a930-1e79-4c09-89d7-abd396442ffc-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:22.330790 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:22.330755    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72" event={"ID":"3a81a930-1e79-4c09-89d7-abd396442ffc","Type":"ContainerDied","Data":"cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99"}
Jul 30 01:31:22.330790 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:22.330768    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"
Jul 30 01:31:22.330790 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:22.330816    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cab15c75595b8157797b89f22c3c45942a09cc9174cbd1e1f773b76e8eeced99"
Jul 30 01:31:23.854241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.854212    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:31:23.886815 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886725    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.886815 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886780    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vcsd\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886837    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886855    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886891    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886919    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886964    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887013 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.886994    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887022    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket\") pod \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\" (UID: \"88ae413a-2e7f-47ed-9d1a-172bfdc08f28\") "
Jul 30 01:31:23.887321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887106    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:23.887321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887234    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:31:23.887480 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887341    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:31:23.887543 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887480    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.887543 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887499    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.887543 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887514    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.887902 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.887869    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:31:23.888188 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.888166    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data" (OuterVolumeSpecName: "istio-data") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:31:23.889579 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.889552    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token" (OuterVolumeSpecName: "istio-token") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:23.889579 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.889568    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd" (OuterVolumeSpecName: "kube-api-access-9vcsd") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "kube-api-access-9vcsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:23.889718 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.889648    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:31:23.889779 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.889757    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "88ae413a-2e7f-47ed-9d1a-172bfdc08f28" (UID: "88ae413a-2e7f-47ed-9d1a-172bfdc08f28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:31:23.988212 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988178    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vcsd\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-kube-api-access-9vcsd\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.988212 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988209    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.988439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988223    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.988439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988235    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.988439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988245    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:23.988439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:23.988256    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88ae413a-2e7f-47ed-9d1a-172bfdc08f28-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:31:24.341568 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.341537    2584 generic.go:356] "Generic (PLEG): container finished" podID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerID="54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c" exitCode=0
Jul 30 01:31:24.341746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.341583    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerDied","Data":"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"}
Jul 30 01:31:24.341746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.341632    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q" event={"ID":"88ae413a-2e7f-47ed-9d1a-172bfdc08f28","Type":"ContainerDied","Data":"7fa0af0f6578408f9652995e57449d74018fdc092c09674745c52fb4960adb94"}
Jul 30 01:31:24.341746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.341588    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"
Jul 30 01:31:24.341746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.341657    2584 scope.go:117] "RemoveContainer" containerID="54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"
Jul 30 01:31:24.351416 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.351397    2584 scope.go:117] "RemoveContainer" containerID="a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"
Jul 30 01:31:24.359394 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.359373    2584 scope.go:117] "RemoveContainer" containerID="54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"
Jul 30 01:31:24.359638 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:31:24.359620    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c\": container with ID starting with 54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c not found: ID does not exist" containerID="54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"
Jul 30 01:31:24.359688 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.359646    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c"} err="failed to get container status \"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c\": rpc error: code = NotFound desc = could not find container \"54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c\": container with ID starting with 54c212e5cc14e1cc2ce903a7f4675c493329cc805a77ef7fa0ffc2d98a89574c not found: ID does not exist"
Jul 30 01:31:24.359688 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.359663    2584 scope.go:117] "RemoveContainer" containerID="a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"
Jul 30 01:31:24.359916 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:31:24.359897    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f\": container with ID starting with a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f not found: ID does not exist" containerID="a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"
Jul 30 01:31:24.359981 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.359926    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f"} err="failed to get container status \"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f\": rpc error: code = NotFound desc = could not find container \"a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f\": container with ID starting with a92ed5d18d6d2effdd12b68d52c892aeecf55bf04a255ab3924fd33e35ab7f3f not found: ID does not exist"
Jul 30 01:31:24.363665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.363643    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:31:24.367477 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:24.367455    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5ca190-openshift-default-6944747694-zwt8q"]
Jul 30 01:31:25.385566 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:25.385465    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" path="/var/lib/kubelet/pods/88ae413a-2e7f-47ed-9d1a-172bfdc08f28/volumes"
Jul 30 01:31:34.608064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:34.608029    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"]
Jul 30 01:31:34.611735 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:34.611711    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b11aa3-f8k72"]
Jul 30 01:31:35.385371 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:35.385327    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a81a930-1e79-4c09-89d7-abd396442ffc" path="/var/lib/kubelet/pods/3a81a930-1e79-4c09-89d7-abd396442ffc/volumes"
Jul 30 01:31:39.545940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:39.545899    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:31:41.917126 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917080    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:31:41.917602 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917587    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:31:41.917645 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917605    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:31:41.917645 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917624    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a81a930-1e79-4c09-89d7-abd396442ffc" containerName="revoke-keys"
Jul 30 01:31:41.917645 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917633    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a81a930-1e79-4c09-89d7-abd396442ffc" containerName="revoke-keys"
Jul 30 01:31:41.917760 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917711    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:31:41.917760 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917721    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:31:41.917760 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.917730    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a81a930-1e79-4c09-89d7-abd396442ffc" containerName="revoke-keys"
Jul 30 01:31:41.920983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.920964    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:41.923471 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.923450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-55b321-openshift-default-dockercfg-6dhdt\""
Jul 30 01:31:41.923560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.923453    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 01:31:41.930573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:41.930547    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:31:42.052197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052158    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052196    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052417 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052217    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052417 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052417 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052289    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052417 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052381    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052550 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052435    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcsk6\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052550 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052475    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.052550 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.052523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.153986 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.153947    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcsk6\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.153996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154115    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154141    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154186 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154266    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154629 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154525    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154629 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154564    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154653    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154675    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.154911 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.154888    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.156535 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.156515    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.156655 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.156636    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.163008 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.162983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.163649 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.163628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcsk6\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6\") pod \"e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") " pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.233581 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.233524    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:42.364378 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.364308    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:31:42.366624 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:31:42.366595    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c8c798_1383_4165_92ad_2a8a2cd7d5b5.slice/crio-8d0fb318dfa520aa73d23199c6f74a0326b1b6ee2829e7bc4a6855bbd21b2334 WatchSource:0}: Error finding container 8d0fb318dfa520aa73d23199c6f74a0326b1b6ee2829e7bc4a6855bbd21b2334: Status 404 returned error can't find the container with id 8d0fb318dfa520aa73d23199c6f74a0326b1b6ee2829e7bc4a6855bbd21b2334
Jul 30 01:31:42.368726 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.368694    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:31:42.368853 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.368759    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:31:42.368853 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.368785    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:31:42.423485 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:42.423453    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerStarted","Data":"8d0fb318dfa520aa73d23199c6f74a0326b1b6ee2829e7bc4a6855bbd21b2334"}
Jul 30 01:31:43.429039 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:43.428998    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerStarted","Data":"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"}
Jul 30 01:31:43.450506 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:43.450455    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podStartSLOduration=2.450439535 podStartE2EDuration="2.450439535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:31:43.448844431 +0000 UTC m=+2890.639792762" watchObservedRunningTime="2026-07-30 01:31:43.450439535 +0000 UTC m=+2890.641387866"
Jul 30 01:31:44.234526 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:44.234487    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:44.239620 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:44.239595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:44.433606 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:44.433571    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:44.434585 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:44.434566    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:31:50.539841 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:50.539806    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:31:59.435745 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.435711    2584 patch_prober.go:28] interesting pod/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 01:31:59.436179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.435766    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 01:31:59.505873 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.505840    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerID="70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52" exitCode=0
Jul 30 01:31:59.506044 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.505915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerDied","Data":"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"}
Jul 30 01:31:59.506279 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.506265    2584 scope.go:117] "RemoveContainer" containerID="70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"
Jul 30 01:31:59.507291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.507265    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:31:59.507349 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.507329    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:31:59.507392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:31:59.507367    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:32:00.536929 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:00.536880    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerStarted","Data":"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"}
Jul 30 01:32:01.233882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:01.233841    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:32:01.234128 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:01.234105    2584 patch_prober.go:28] interesting pod/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 01:32:01.234174 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:01.234154    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 01:32:02.234049 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:02.234009    2584 patch_prober.go:28] interesting pod/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 01:32:02.234428 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:02.234079    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 01:32:03.234160 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:03.234128    2584 patch_prober.go:28] interesting pod/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 01:32:03.234530 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:03.234196    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 01:32:04.234075 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:04.234035    2584 patch_prober.go:28] interesting pod/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 01:32:04.234291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:04.234110    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 01:32:05.237847 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:05.237814    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:32:05.238292 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:05.238072    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:32:05.238779 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:05.238761    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:32:22.146046 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:22.145990    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:32:45.951642 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.951603    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:32:45.952132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.951939    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:32:45.952132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.951951    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ae413a-2e7f-47ed-9d1a-172bfdc08f28" containerName="istio-proxy"
Jul 30 01:32:45.954010 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.953992    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:45.957422 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.957396    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"kube-root-ca.crt\""
Jul 30 01:32:45.957564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.957425    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"openshift-service-ca.crt\""
Jul 30 01:32:45.957564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.957432    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"default-dockercfg-8chk5\""
Jul 30 01:32:45.957564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.957497    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"auth-test-model-55b321-kserve-self-signed-certs\""
Jul 30 01:32:45.967049 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:45.967026    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:32:46.118092 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.118270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118107    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6k2z\" (UniqueName: \"kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.118270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.118270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118164    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.118270 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118224    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.118409 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.118332    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219236 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219150    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219236 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6k2z\" (UniqueName: \"kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219236 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219219    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219236 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219578 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219255    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219578 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219721 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219692    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219779 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219727    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.219779 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.219739    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.221513 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.221493    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.221699 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.221682    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.226858 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.226837    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6k2z\" (UniqueName: \"kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z\") pod \"auth-test-model-55b321-kserve-5f66d9574c-kx6fn\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") " pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.262981 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.262955    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:46.390146 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.390120    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:32:46.392033 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:32:46.392003    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f480f4d_081f_4dbe_b75e_4b67d8b37131.slice/crio-a193571152080ed63f7d545a6d48730a77b599add23531e095438749a90b3566 WatchSource:0}: Error finding container a193571152080ed63f7d545a6d48730a77b599add23531e095438749a90b3566: Status 404 returned error can't find the container with id a193571152080ed63f7d545a6d48730a77b599add23531e095438749a90b3566
Jul 30 01:32:46.747223 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.747187    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" event={"ID":"4f480f4d-081f-4dbe-b75e-4b67d8b37131","Type":"ContainerStarted","Data":"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"}
Jul 30 01:32:46.747223 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.747226    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" event={"ID":"4f480f4d-081f-4dbe-b75e-4b67d8b37131","Type":"ContainerStarted","Data":"a193571152080ed63f7d545a6d48730a77b599add23531e095438749a90b3566"}
Jul 30 01:32:46.763641 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:46.763590    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" podStartSLOduration=1.763575667 podStartE2EDuration="1.763575667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:32:46.762196477 +0000 UTC m=+2953.953144809" watchObservedRunningTime="2026-07-30 01:32:46.763575667 +0000 UTC m=+2953.954524033"
Jul 30 01:32:49.240542 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:49.240508    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:32:56.264100 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:56.264007    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:56.264100 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:56.264068    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:56.276597 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:56.276564    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:32:56.804467 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:32:56.804433    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:33:19.650143 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.650114    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:33:19.653486 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.653461    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.656963 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.656937    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"kube-root-ca.crt\""
Jul 30 01:33:19.657105 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.656967    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"default-dockercfg-67bfj\""
Jul 30 01:33:19.657105 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.656978    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"openshift-service-ca.crt\""
Jul 30 01:33:19.657105 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.656984    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"auth-test-model-fba93c-kserve-self-signed-certs\""
Jul 30 01:33:19.664378 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.664346    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:33:19.680202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680166    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.680202 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680206    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.680426 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.680426 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.680426 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpq9v\" (UniqueName: \"kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.680426 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.680403    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781540 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781591    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.781720 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpq9v\" (UniqueName: \"kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.782002 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.781987    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.782063 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.782015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.782121 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.782100    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.783899 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.783880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.784179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.784162    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.790470 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.790447    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpq9v\" (UniqueName: \"kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v\") pod \"auth-test-model-fba93c-kserve-c88f69f9f-nwcvc\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") " pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:19.964530 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:19.964431    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:20.096042 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:20.096001    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:33:20.102688 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:33:20.099226    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab179a82_4035_449a_b60f_b95c2b7d31a1.slice/crio-28caae9eaff8febc51d1522bd33ddc94933b5ce84151792185f43d178fcf3279 WatchSource:0}: Error finding container 28caae9eaff8febc51d1522bd33ddc94933b5ce84151792185f43d178fcf3279: Status 404 returned error can't find the container with id 28caae9eaff8febc51d1522bd33ddc94933b5ce84151792185f43d178fcf3279
Jul 30 01:33:20.906903 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:20.906860    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" event={"ID":"ab179a82-4035-449a-b60f-b95c2b7d31a1","Type":"ContainerStarted","Data":"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"}
Jul 30 01:33:20.906903 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:20.906903    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" event={"ID":"ab179a82-4035-449a-b60f-b95c2b7d31a1","Type":"ContainerStarted","Data":"28caae9eaff8febc51d1522bd33ddc94933b5ce84151792185f43d178fcf3279"}
Jul 30 01:33:20.923352 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:20.923301    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" podStartSLOduration=1.923288024 podStartE2EDuration="1.923288024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:33:20.922078552 +0000 UTC m=+2988.113026884" watchObservedRunningTime="2026-07-30 01:33:20.923288024 +0000 UTC m=+2988.114236418"
Jul 30 01:33:23.538910 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:23.538873    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:33:29.965227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:29.965192    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:29.965227 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:29.965236    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:29.978046 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:29.978023    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:30.964928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:30.964896    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:33:33.634523 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:33.634492    2584 scope.go:117] "RemoveContainer" containerID="350ce63ddd154a20e9a14bd2116b62d65d5422720c46e27145f4f99eacaf9d28"
Jul 30 01:33:55.752706 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:33:55.752671    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:34:21.635855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:34:21.635821    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:34:38.445824 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:34:38.445722    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:35:05.640717 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:35:05.640681    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:35:23.360783 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:35:23.355747    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:35:50.749023 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:35:50.748988    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:36:07.348242 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:36:07.348142    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:36:33.666813 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:36:33.666763    2584 scope.go:117] "RemoveContainer" containerID="7c790e6524c3f058970ae4fb25efd7ced8341c576042e324c589b9726f694335"
Jul 30 01:36:34.443044 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:36:34.443007    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:36:51.036042 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:36:51.036001    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:37:19.039177 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:37:19.039140    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:37:33.691303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:37:33.691222    2584 scope.go:117] "RemoveContainer" containerID="c9a076d37e835bafa53205fc4a6102427d574c25644175610d50cbae6d094ac7"
Jul 30 01:37:33.699487 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:37:33.699468    2584 scope.go:117] "RemoveContainer" containerID="bbba76e1ce5a0b3d5614d5e4da9955f7ac6c844d4ef492bd073707e9a2088dea"
Jul 30 01:37:37.948218 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:37:37.948185    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:38:04.740030 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:04.739998    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:38:09.637856 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:09.637818    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:38:10.582071 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:10.582037    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:38:22.448082 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:22.448049    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:38:24.648193 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.648162    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:38:24.651495 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.651478    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.653998 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.653972    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"sub-test-model-55b321-kserve-self-signed-certs\""
Jul 30 01:38:24.658769 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.658724    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" podUID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" containerName="main" containerID="cri-o://594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935" gracePeriod=45
Jul 30 01:38:24.659392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.659301    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:38:24.772626 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772595    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.772787 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772635    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.772787 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.772944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772775    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbkml\" (UniqueName: \"kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.772944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772897    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.773040 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.772962    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873551 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.873861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.873728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbkml\" (UniqueName: \"kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.874184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.874164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.874244 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.874227    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.874301 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.874259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.875971 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.875950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.876389 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.876370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.882442 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.882418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbkml\" (UniqueName: \"kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml\") pod \"sub-test-model-55b321-kserve-54f5947d7-qlxsq\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") " pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.907011 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.906956    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:38:24.962022 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.961983    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:24.974848 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.974820    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.974968 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.974869    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.974968 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.974918    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.975089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.974973    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6k2z\" (UniqueName: \"kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.975089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975052    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.975205 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975117    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache\") pod \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\" (UID: \"4f480f4d-081f-4dbe-b75e-4b67d8b37131\") "
Jul 30 01:38:24.975604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975293    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home" (OuterVolumeSpecName: "home") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:24.975604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975400    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:24.975604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975478    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:24.975604 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.975547    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:24.977310 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.977287    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:38:24.977310 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.977300    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z" (OuterVolumeSpecName: "kube-api-access-r6k2z") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "kube-api-access-r6k2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:38:24.977456 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:24.977370    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm" (OuterVolumeSpecName: "dshm") pod "4f480f4d-081f-4dbe-b75e-4b67d8b37131" (UID: "4f480f4d-081f-4dbe-b75e-4b67d8b37131"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:25.076414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.076378    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6k2z\" (UniqueName: \"kubernetes.io/projected/4f480f4d-081f-4dbe-b75e-4b67d8b37131-kube-api-access-r6k2z\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.076414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.076409    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.076414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.076419    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.076641 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.076431    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f480f4d-081f-4dbe-b75e-4b67d8b37131-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.076641 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.076439    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f480f4d-081f-4dbe-b75e-4b67d8b37131-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.093346 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.093319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:38:25.095161 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:38:25.095134    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cdfce5f_921c_4a4e_b9ce_6eabed8ba0fc.slice/crio-6fe0ef1622a517521ecc5c87bb0b69c3052a1a578385c7f9df7a14e64095f3b7 WatchSource:0}: Error finding container 6fe0ef1622a517521ecc5c87bb0b69c3052a1a578385c7f9df7a14e64095f3b7: Status 404 returned error can't find the container with id 6fe0ef1622a517521ecc5c87bb0b69c3052a1a578385c7f9df7a14e64095f3b7
Jul 30 01:38:25.326584 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.326544    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" event={"ID":"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc","Type":"ContainerStarted","Data":"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"}
Jul 30 01:38:25.326780 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.326596    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" event={"ID":"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc","Type":"ContainerStarted","Data":"6fe0ef1622a517521ecc5c87bb0b69c3052a1a578385c7f9df7a14e64095f3b7"}
Jul 30 01:38:25.327830 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.327787    2584 generic.go:356] "Generic (PLEG): container finished" podID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" containerID="594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935" exitCode=0
Jul 30 01:38:25.327915 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.327851    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"
Jul 30 01:38:25.327915 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.327866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" event={"ID":"4f480f4d-081f-4dbe-b75e-4b67d8b37131","Type":"ContainerDied","Data":"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"}
Jul 30 01:38:25.327915 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.327908    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn" event={"ID":"4f480f4d-081f-4dbe-b75e-4b67d8b37131","Type":"ContainerDied","Data":"a193571152080ed63f7d545a6d48730a77b599add23531e095438749a90b3566"}
Jul 30 01:38:25.328018 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.327925    2584 scope.go:117] "RemoveContainer" containerID="594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"
Jul 30 01:38:25.336605 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.336582    2584 scope.go:117] "RemoveContainer" containerID="594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"
Jul 30 01:38:25.336895 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:38:25.336874    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935\": container with ID starting with 594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935 not found: ID does not exist" containerID="594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"
Jul 30 01:38:25.336976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.336904    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935"} err="failed to get container status \"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935\": rpc error: code = NotFound desc = could not find container \"594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935\": container with ID starting with 594cfd84223d7be164c9e0be4608e11e3aae21639338cc57035317d46aee7935 not found: ID does not exist"
Jul 30 01:38:25.346318 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.346274    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" podStartSLOduration=1.346259473 podStartE2EDuration="1.346259473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:38:25.345687477 +0000 UTC m=+3292.536635809" watchObservedRunningTime="2026-07-30 01:38:25.346259473 +0000 UTC m=+3292.537207781"
Jul 30 01:38:25.358533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.358497    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:38:25.364092 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.364064    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/auth-test-model-55b321-kserve-5f66d9574c-kx6fn"]
Jul 30 01:38:25.386286 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.386253    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" path="/var/lib/kubelet/pods/4f480f4d-081f-4dbe-b75e-4b67d8b37131/volumes"
Jul 30 01:38:25.600891 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.600777    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" podUID="ab179a82-4035-449a-b60f-b95c2b7d31a1" containerName="main" containerID="cri-o://105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929" gracePeriod=45
Jul 30 01:38:25.857569 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.857512    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:38:25.986308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986272    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986327    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986351    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986369    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986403    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986464    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpq9v\" (UniqueName: \"kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v\") pod \"ab179a82-4035-449a-b60f-b95c2b7d31a1\" (UID: \"ab179a82-4035-449a-b60f-b95c2b7d31a1\") "
Jul 30 01:38:25.986746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986647    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:25.986746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986660    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:25.986746 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986695    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home" (OuterVolumeSpecName: "home") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:25.986928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986835    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.986928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986849    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.986928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.986858    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:25.988887 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.988847    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v" (OuterVolumeSpecName: "kube-api-access-xpq9v") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "kube-api-access-xpq9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:38:25.988887 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.988858    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:38:25.989029 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:25.988864    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm" (OuterVolumeSpecName: "dshm") pod "ab179a82-4035-449a-b60f-b95c2b7d31a1" (UID: "ab179a82-4035-449a-b60f-b95c2b7d31a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:38:26.088177 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.088125    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab179a82-4035-449a-b60f-b95c2b7d31a1-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:26.088177 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.088173    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab179a82-4035-449a-b60f-b95c2b7d31a1-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:26.088177 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.088185    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpq9v\" (UniqueName: \"kubernetes.io/projected/ab179a82-4035-449a-b60f-b95c2b7d31a1-kube-api-access-xpq9v\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:38:26.339912 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.339874    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab179a82-4035-449a-b60f-b95c2b7d31a1" containerID="105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929" exitCode=0
Jul 30 01:38:26.340113 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.339934    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"
Jul 30 01:38:26.340113 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.339955    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" event={"ID":"ab179a82-4035-449a-b60f-b95c2b7d31a1","Type":"ContainerDied","Data":"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"}
Jul 30 01:38:26.340113 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.339994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc" event={"ID":"ab179a82-4035-449a-b60f-b95c2b7d31a1","Type":"ContainerDied","Data":"28caae9eaff8febc51d1522bd33ddc94933b5ce84151792185f43d178fcf3279"}
Jul 30 01:38:26.340113 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.340018    2584 scope.go:117] "RemoveContainer" containerID="105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"
Jul 30 01:38:26.349647 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.349627    2584 scope.go:117] "RemoveContainer" containerID="105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"
Jul 30 01:38:26.349993 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:38:26.349971    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929\": container with ID starting with 105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929 not found: ID does not exist" containerID="105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"
Jul 30 01:38:26.350100 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.349999    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929"} err="failed to get container status \"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929\": rpc error: code = NotFound desc = could not find container \"105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929\": container with ID starting with 105ad660f1e4d6c5a6b2329f2c61f7332bc7f131c95a9257d54ee108e88a4929 not found: ID does not exist"
Jul 30 01:38:26.361300 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.361269    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:38:26.365318 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:26.365287    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/auth-test-model-fba93c-kserve-c88f69f9f-nwcvc"]
Jul 30 01:38:27.384960 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:27.384923    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab179a82-4035-449a-b60f-b95c2b7d31a1" path="/var/lib/kubelet/pods/ab179a82-4035-449a-b60f-b95c2b7d31a1/volumes"
Jul 30 01:38:34.962653 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:34.962615    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:34.962653 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:34.962666    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:34.975503 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:34.975475    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:35.395922 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:35.395893    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:38:35.543818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:35.543770    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:38:58.844915 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.844830    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:38:58.845359 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845339    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab179a82-4035-449a-b60f-b95c2b7d31a1" containerName="main"
Jul 30 01:38:58.845403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845364    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab179a82-4035-449a-b60f-b95c2b7d31a1" containerName="main"
Jul 30 01:38:58.845403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845383    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" containerName="main"
Jul 30 01:38:58.845403 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845392    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" containerName="main"
Jul 30 01:38:58.845511 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845499    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab179a82-4035-449a-b60f-b95c2b7d31a1" containerName="main"
Jul 30 01:38:58.845556 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.845516    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f480f4d-081f-4dbe-b75e-4b67d8b37131" containerName="main"
Jul 30 01:38:58.848985 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.848965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.852326 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.852305    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"kube-root-ca.crt\""
Jul 30 01:38:58.853240 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.853220    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"openshift-service-ca.crt\""
Jul 30 01:38:58.853350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.853278    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"sub-test-model-fba93c-kserve-self-signed-certs\""
Jul 30 01:38:58.853414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.853331    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"default-dockercfg-67bfj\""
Jul 30 01:38:58.865839 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.865816    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:38:58.892458 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892432    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7tb5\" (UniqueName: \"kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.892550 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892487    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.892659 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.892659 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892603    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.892659 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892622    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.892831 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.892708    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.993964 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.993928    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7tb5\" (UniqueName: \"kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.993983    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994014    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994049    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994078    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994464 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994438    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994534 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994473    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.994534 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.994503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.996534 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.996517    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:58.996611 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:58.996594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:59.006894 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.006866    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7tb5\" (UniqueName: \"kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5\") pod \"sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") " pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:59.163818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.158443    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:38:59.292646 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.292620    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:38:59.295005 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:38:59.294985    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02311dda_a6b6_4bca_b540_6f21059d6810.slice/crio-90eedfda899d08cfa88a1172c71cf4488fa64eb505fb3578b85bd866e50c6202 WatchSource:0}: Error finding container 90eedfda899d08cfa88a1172c71cf4488fa64eb505fb3578b85bd866e50c6202: Status 404 returned error can't find the container with id 90eedfda899d08cfa88a1172c71cf4488fa64eb505fb3578b85bd866e50c6202
Jul 30 01:38:59.491954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.491914    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" event={"ID":"02311dda-a6b6-4bca-b540-6f21059d6810","Type":"ContainerStarted","Data":"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"}
Jul 30 01:38:59.491954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.491959    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" event={"ID":"02311dda-a6b6-4bca-b540-6f21059d6810","Type":"ContainerStarted","Data":"90eedfda899d08cfa88a1172c71cf4488fa64eb505fb3578b85bd866e50c6202"}
Jul 30 01:38:59.510476 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:38:59.510428    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" podStartSLOduration=1.5104126469999999 podStartE2EDuration="1.510412647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:38:59.508664608 +0000 UTC m=+3326.699612938" watchObservedRunningTime="2026-07-30 01:38:59.510412647 +0000 UTC m=+3326.701360977"
Jul 30 01:39:02.343456 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:02.343420    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:39:09.159052 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:09.159013    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:39:09.159573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:09.159064    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:39:09.171274 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:09.171241    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:39:09.547680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:09.547654    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:39:49.537395 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:39:49.537353    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:40:07.243452 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:07.243413    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:40:30.548222 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:30.548148    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:40:36.249205 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:36.249162    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:40:37.180442 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:37.180402    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:40:48.743809 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:48.743766    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:40:51.251075 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.251040    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:40:51.254364 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.254346    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.256581 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.256559    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-55b321\"/\"rate-test-model-55b321-kserve-self-signed-certs\""
Jul 30 01:40:51.263815 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.263776    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:40:51.267851 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.267788    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" podUID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" containerName="main" containerID="cri-o://b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818" gracePeriod=45
Jul 30 01:40:51.409750 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409712    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.409750 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409755    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp4fq\" (UniqueName: \"kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.410006 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409786    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.410006 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.410006 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.410006 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.409907    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511378 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511512 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511387    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511512 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511635 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511544    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511635 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp4fq\" (UniqueName: \"kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511747 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.511885 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.511840    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.512120 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.512097    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.513648 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.513628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.513828 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.513791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.519507 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.519485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp4fq\" (UniqueName: \"kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq\") pod \"rate-test-model-55b321-kserve-6fc47864b7-lz2sg\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") " pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.536775 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.536758    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:40:51.563923 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.563895    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:40:51.694301 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.694161    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:40:51.696589 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:40:51.696561    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod747214a1_7902_45c3_9ebd_1f6d9019aa49.slice/crio-c1e215ab734197a8596751400c475a829be8d192709be7d17f40c676f2d977dc WatchSource:0}: Error finding container c1e215ab734197a8596751400c475a829be8d192709be7d17f40c676f2d977dc: Status 404 returned error can't find the container with id c1e215ab734197a8596751400c475a829be8d192709be7d17f40c676f2d977dc
Jul 30 01:40:51.713303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713283    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713313    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713347    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713555 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713434    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713555 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713467    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbkml\" (UniqueName: \"kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713555 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713516    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache\") pod \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\" (UID: \"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc\") "
Jul 30 01:40:51.713711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713580    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home" (OuterVolumeSpecName: "home") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:51.713811 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713740    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:51.713811 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713786    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:51.713933 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.713821    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:51.715703 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.715668    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml" (OuterVolumeSpecName: "kube-api-access-zbkml") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "kube-api-access-zbkml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:40:51.715849 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.715830    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:40:51.715916 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.715885    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm" (OuterVolumeSpecName: "dshm") pod "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" (UID: "9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:51.814308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.814234    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:51.814308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.814261    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbkml\" (UniqueName: \"kubernetes.io/projected/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-kube-api-access-zbkml\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:51.814308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.814273    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:51.814308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.814286    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:51.814308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:51.814298    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.021288 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.021245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" event={"ID":"747214a1-7902-45c3-9ebd-1f6d9019aa49","Type":"ContainerStarted","Data":"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"}
Jul 30 01:40:52.021288 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.021292    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" event={"ID":"747214a1-7902-45c3-9ebd-1f6d9019aa49","Type":"ContainerStarted","Data":"c1e215ab734197a8596751400c475a829be8d192709be7d17f40c676f2d977dc"}
Jul 30 01:40:52.022513 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.022491    2584 generic.go:356] "Generic (PLEG): container finished" podID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" containerID="b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818" exitCode=0
Jul 30 01:40:52.022624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.022526    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"
Jul 30 01:40:52.022624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.022551    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" event={"ID":"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc","Type":"ContainerDied","Data":"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"}
Jul 30 01:40:52.022624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.022578    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq" event={"ID":"9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc","Type":"ContainerDied","Data":"6fe0ef1622a517521ecc5c87bb0b69c3052a1a578385c7f9df7a14e64095f3b7"}
Jul 30 01:40:52.022624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.022595    2584 scope.go:117] "RemoveContainer" containerID="b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"
Jul 30 01:40:52.031265 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.031249    2584 scope.go:117] "RemoveContainer" containerID="b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"
Jul 30 01:40:52.031508 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:40:52.031489    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818\": container with ID starting with b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818 not found: ID does not exist" containerID="b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"
Jul 30 01:40:52.031573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.031515    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818"} err="failed to get container status \"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818\": rpc error: code = NotFound desc = could not find container \"b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818\": container with ID starting with b953331dd76d59ae26d791d7111e89c5a9a21e67d3b15e550f7ab56461128818 not found: ID does not exist"
Jul 30 01:40:52.037043 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.037001    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" podStartSLOduration=1.036989066 podStartE2EDuration="1.036989066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:40:52.035450232 +0000 UTC m=+3439.226398563" watchObservedRunningTime="2026-07-30 01:40:52.036989066 +0000 UTC m=+3439.227937414"
Jul 30 01:40:52.046831 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.046785    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:40:52.050101 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.050082    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/sub-test-model-55b321-kserve-54f5947d7-qlxsq"]
Jul 30 01:40:52.199421 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.199334    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" podUID="02311dda-a6b6-4bca-b540-6f21059d6810" containerName="main" containerID="cri-o://49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255" gracePeriod=45
Jul 30 01:40:52.452077 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.452018    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:40:52.622149 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622116    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622193    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7tb5\" (UniqueName: \"kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622225    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622244    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622270    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622549 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622352    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm\") pod \"02311dda-a6b6-4bca-b540-6f21059d6810\" (UID: \"02311dda-a6b6-4bca-b540-6f21059d6810\") "
Jul 30 01:40:52.622549 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622534    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:52.622660 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622548    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home" (OuterVolumeSpecName: "home") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:52.622660 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622558    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache" (OuterVolumeSpecName: "model-cache") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:52.622733 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622676    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.622733 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622691    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.622733 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.622710    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.624372 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.624322    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:40:52.624459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.624392    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5" (OuterVolumeSpecName: "kube-api-access-p7tb5") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "kube-api-access-p7tb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:40:52.624459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.624417    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm" (OuterVolumeSpecName: "dshm") pod "02311dda-a6b6-4bca-b540-6f21059d6810" (UID: "02311dda-a6b6-4bca-b540-6f21059d6810"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:40:52.723326 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.723270    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7tb5\" (UniqueName: \"kubernetes.io/projected/02311dda-a6b6-4bca-b540-6f21059d6810-kube-api-access-p7tb5\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.723326 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.723293    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02311dda-a6b6-4bca-b540-6f21059d6810-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:52.723326 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:52.723302    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02311dda-a6b6-4bca-b540-6f21059d6810-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:40:53.029625 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.029587    2584 generic.go:356] "Generic (PLEG): container finished" podID="02311dda-a6b6-4bca-b540-6f21059d6810" containerID="49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255" exitCode=0
Jul 30 01:40:53.029819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.029629    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"
Jul 30 01:40:53.029819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.029671    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" event={"ID":"02311dda-a6b6-4bca-b540-6f21059d6810","Type":"ContainerDied","Data":"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"}
Jul 30 01:40:53.029819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.029705    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k" event={"ID":"02311dda-a6b6-4bca-b540-6f21059d6810","Type":"ContainerDied","Data":"90eedfda899d08cfa88a1172c71cf4488fa64eb505fb3578b85bd866e50c6202"}
Jul 30 01:40:53.029819 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.029726    2584 scope.go:117] "RemoveContainer" containerID="49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"
Jul 30 01:40:53.038566 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.038547    2584 scope.go:117] "RemoveContainer" containerID="49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"
Jul 30 01:40:53.038835 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:40:53.038817    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255\": container with ID starting with 49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255 not found: ID does not exist" containerID="49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"
Jul 30 01:40:53.038886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.038843    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255"} err="failed to get container status \"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255\": rpc error: code = NotFound desc = could not find container \"49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255\": container with ID starting with 49285e6cf445b48b594e3960c8f5dc6d866524cfb540ad8bba54ea31a3913255 not found: ID does not exist"
Jul 30 01:40:53.050259 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.050233    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:40:53.054368 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.054345    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/sub-test-model-fba93c-kserve-7b6d75b6c-2sp7k"]
Jul 30 01:40:53.385666 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.385583    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02311dda-a6b6-4bca-b540-6f21059d6810" path="/var/lib/kubelet/pods/02311dda-a6b6-4bca-b540-6f21059d6810/volumes"
Jul 30 01:40:53.386015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:53.386000    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" path="/var/lib/kubelet/pods/9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc/volumes"
Jul 30 01:40:55.842081 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:40:55.842049    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:41:01.564533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:01.564497    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:41:01.564964 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:01.564549    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:41:01.576965 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:01.576939    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:41:02.085132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:02.085102    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:41:07.047531 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047499    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:41:07.047927 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047842    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02311dda-a6b6-4bca-b540-6f21059d6810" containerName="main"
Jul 30 01:41:07.047927 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047854    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="02311dda-a6b6-4bca-b540-6f21059d6810" containerName="main"
Jul 30 01:41:07.047927 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047883    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" containerName="main"
Jul 30 01:41:07.047927 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047888    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" containerName="main"
Jul 30 01:41:07.048069 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047944    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="02311dda-a6b6-4bca-b540-6f21059d6810" containerName="main"
Jul 30 01:41:07.048069 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.047955    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cdfce5f-921c-4a4e-b9ce-6eabed8ba0fc" containerName="main"
Jul 30 01:41:07.052319 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.052298    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.054649 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.054627    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"kube-root-ca.crt\""
Jul 30 01:41:07.054822 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.054786    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"openshift-service-ca.crt\""
Jul 30 01:41:07.056249 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.056199    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"default-dockercfg-67bfj\""
Jul 30 01:41:07.056412 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.056195    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fba93c\"/\"rate-test-model-fba93c-kserve-self-signed-certs\""
Jul 30 01:41:07.063255 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.062954    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:41:07.147517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147486    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.147724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ngh6\" (UniqueName: \"kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.147724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147572    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.147724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.147724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147680    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.147925 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.147757    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.248682 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.248682 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248685    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249062 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249062 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248734    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249062 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248774    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249062 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.248791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ngh6\" (UniqueName: \"kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.249123    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.249210    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.249271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.249225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.251127 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.251101    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.251365 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.251347    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.256626 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.256599    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ngh6\" (UniqueName: \"kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6\") pod \"rate-test-model-fba93c-kserve-6c9b54bc75-z9dks\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") " pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.366081 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.365992    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:07.498381 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:07.498270    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:41:07.500919 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:41:07.500889    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff3882a4_3b77_4d18_8bb9_72bbacd97e43.slice/crio-542c508b55fb433e4126a0729e90f3532e43232e4065b4966f867d3f7b790a43 WatchSource:0}: Error finding container 542c508b55fb433e4126a0729e90f3532e43232e4065b4966f867d3f7b790a43: Status 404 returned error can't find the container with id 542c508b55fb433e4126a0729e90f3532e43232e4065b4966f867d3f7b790a43
Jul 30 01:41:08.101655 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:08.101614    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" event={"ID":"ff3882a4-3b77-4d18-8bb9-72bbacd97e43","Type":"ContainerStarted","Data":"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"}
Jul 30 01:41:08.101655 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:08.101657    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" event={"ID":"ff3882a4-3b77-4d18-8bb9-72bbacd97e43","Type":"ContainerStarted","Data":"542c508b55fb433e4126a0729e90f3532e43232e4065b4966f867d3f7b790a43"}
Jul 30 01:41:08.117114 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:08.117070    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" podStartSLOduration=1.117055125 podStartE2EDuration="1.117055125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:41:08.116405288 +0000 UTC m=+3455.307353658" watchObservedRunningTime="2026-07-30 01:41:08.117055125 +0000 UTC m=+3455.308003455"
Jul 30 01:41:10.545445 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:10.545406    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:41:17.366739 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:17.366697    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:17.367171 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:17.366762    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:17.379209 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:17.379178    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:41:18.158756 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:41:18.158726    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:42:01.248448 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:01.248368    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:42:20.436916 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:20.436880    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:42:42.035010 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:42.034976    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:42:48.747041 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:48.747005    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:42:49.929867 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:49.929826    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:42:50.863598 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:50.860943    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531163056"]
Jul 30 01:42:50.865362 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:50.864819    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:50.868540 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:50.868512    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jdhmx\""
Jul 30 01:42:50.870817 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:50.870769    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531163056"]
Jul 30 01:42:50.967755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:50.967723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdd9k\" (UniqueName: \"kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k\") pod \"test-curl-7255-139727531163056\" (UID: \"13413a10-d706-44c0-bfae-88a523e4a55f\") " pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:51.068274 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.068238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdd9k\" (UniqueName: \"kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k\") pod \"test-curl-7255-139727531163056\" (UID: \"13413a10-d706-44c0-bfae-88a523e4a55f\") " pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:51.076152 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.076132    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdd9k\" (UniqueName: \"kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k\") pod \"test-curl-7255-139727531163056\" (UID: \"13413a10-d706-44c0-bfae-88a523e4a55f\") " pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:51.176617 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.176539    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:51.306764 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.306735    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531163056"]
Jul 30 01:42:51.308359 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:42:51.308319    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13413a10_d706_44c0_bfae_88a523e4a55f.slice/crio-1f50d8076494e41e9cd1b47370e09b9d4c2e85af9e179bae5a8edcd9f4ec43ee WatchSource:0}: Error finding container 1f50d8076494e41e9cd1b47370e09b9d4c2e85af9e179bae5a8edcd9f4ec43ee: Status 404 returned error can't find the container with id 1f50d8076494e41e9cd1b47370e09b9d4c2e85af9e179bae5a8edcd9f4ec43ee
Jul 30 01:42:51.309816 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.309777    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:42:51.578165 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:51.578124    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531163056" event={"ID":"13413a10-d706-44c0-bfae-88a523e4a55f","Type":"ContainerStarted","Data":"1f50d8076494e41e9cd1b47370e09b9d4c2e85af9e179bae5a8edcd9f4ec43ee"}
Jul 30 01:42:52.584531 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:52.584500    2584 generic.go:356] "Generic (PLEG): container finished" podID="13413a10-d706-44c0-bfae-88a523e4a55f" containerID="c10072bc8974492e51fb3d895c42e50786fd26f3cdb135ee1c5dd688080c9f6c" exitCode=0
Jul 30 01:42:52.584961 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:52.584583    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531163056" event={"ID":"13413a10-d706-44c0-bfae-88a523e4a55f","Type":"ContainerDied","Data":"c10072bc8974492e51fb3d895c42e50786fd26f3cdb135ee1c5dd688080c9f6c"}
Jul 30 01:42:53.723118 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.723094    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:53.754949 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.754922    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531163056_13413a10-d706-44c0-bfae-88a523e4a55f/test-curl-7255-139727531163056/0.log"
Jul 30 01:42:53.776066 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.776033    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531163056"]
Jul 30 01:42:53.780158 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.780135    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531163056"]
Jul 30 01:42:53.791653 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.791631    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdd9k\" (UniqueName: \"kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k\") pod \"13413a10-d706-44c0-bfae-88a523e4a55f\" (UID: \"13413a10-d706-44c0-bfae-88a523e4a55f\") "
Jul 30 01:42:53.793880 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.793860    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k" (OuterVolumeSpecName: "kube-api-access-qdd9k") pod "13413a10-d706-44c0-bfae-88a523e4a55f" (UID: "13413a10-d706-44c0-bfae-88a523e4a55f"). InnerVolumeSpecName "kube-api-access-qdd9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:42:53.892667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.892579    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdd9k\" (UniqueName: \"kubernetes.io/projected/13413a10-d706-44c0-bfae-88a523e4a55f-kube-api-access-qdd9k\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:42:53.902008 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.901978    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531079216"]
Jul 30 01:42:53.902333 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.902320    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13413a10-d706-44c0-bfae-88a523e4a55f" containerName="test-curl-7255-139727531163056"
Jul 30 01:42:53.902375 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.902336    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="13413a10-d706-44c0-bfae-88a523e4a55f" containerName="test-curl-7255-139727531163056"
Jul 30 01:42:53.902411 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.902398    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="13413a10-d706-44c0-bfae-88a523e4a55f" containerName="test-curl-7255-139727531163056"
Jul 30 01:42:53.904219 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.904202    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:53.911708 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.911666    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531079216"]
Jul 30 01:42:53.993650 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:53.993610    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzq97\" (UniqueName: \"kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97\") pod \"test-curl-7255-139727531079216\" (UID: \"7e9f84bf-337f-401d-9e14-802a88a14e20\") " pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:54.095026 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.094990    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzq97\" (UniqueName: \"kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97\") pod \"test-curl-7255-139727531079216\" (UID: \"7e9f84bf-337f-401d-9e14-802a88a14e20\") " pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:54.106976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.106941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzq97\" (UniqueName: \"kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97\") pod \"test-curl-7255-139727531079216\" (UID: \"7e9f84bf-337f-401d-9e14-802a88a14e20\") " pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:54.214572 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.214473    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:54.342349 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.342324    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531079216"]
Jul 30 01:42:54.343994 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:42:54.343968    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e9f84bf_337f_401d_9e14_802a88a14e20.slice/crio-adfb70ea46fa45f7a89ae56590ef869f787e854a7a5c7a52f3781accac6511fc WatchSource:0}: Error finding container adfb70ea46fa45f7a89ae56590ef869f787e854a7a5c7a52f3781accac6511fc: Status 404 returned error can't find the container with id adfb70ea46fa45f7a89ae56590ef869f787e854a7a5c7a52f3781accac6511fc
Jul 30 01:42:54.601376 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.601337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531079216" event={"ID":"7e9f84bf-337f-401d-9e14-802a88a14e20","Type":"ContainerStarted","Data":"adfb70ea46fa45f7a89ae56590ef869f787e854a7a5c7a52f3781accac6511fc"}
Jul 30 01:42:54.602478 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.602453    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f50d8076494e41e9cd1b47370e09b9d4c2e85af9e179bae5a8edcd9f4ec43ee"
Jul 30 01:42:54.602478 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:54.602468    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:54.613002 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:42:54.612973    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7255-139727531163056\" is forbidden: User \"system:node:ip-10-0-134-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-28.ec2.internal' and this object" podUID="13413a10-d706-44c0-bfae-88a523e4a55f" pod="opendatahub/test-curl-7255-139727531163056"
Jul 30 01:42:55.386787 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:55.386748    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13413a10-d706-44c0-bfae-88a523e4a55f" path="/var/lib/kubelet/pods/13413a10-d706-44c0-bfae-88a523e4a55f/volumes"
Jul 30 01:42:55.609237 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:55.609205    2584 generic.go:356] "Generic (PLEG): container finished" podID="7e9f84bf-337f-401d-9e14-802a88a14e20" containerID="32c2c7dc632bd030b572d018dd5a2f9a81b47d10c059fc4cf27cb95598aa0fd3" exitCode=0
Jul 30 01:42:55.609237 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:55.609248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531079216" event={"ID":"7e9f84bf-337f-401d-9e14-802a88a14e20","Type":"ContainerDied","Data":"32c2c7dc632bd030b572d018dd5a2f9a81b47d10c059fc4cf27cb95598aa0fd3"}
Jul 30 01:42:56.745759 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.745737    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:56.764130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.764105    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531079216_7e9f84bf-337f-401d-9e14-802a88a14e20/test-curl-7255-139727531079216/0.log"
Jul 30 01:42:56.784066 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.784037    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531079216"]
Jul 30 01:42:56.786281 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.786256    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531079216"]
Jul 30 01:42:56.819658 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.819629    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzq97\" (UniqueName: \"kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97\") pod \"7e9f84bf-337f-401d-9e14-802a88a14e20\" (UID: \"7e9f84bf-337f-401d-9e14-802a88a14e20\") "
Jul 30 01:42:56.821705 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.821682    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97" (OuterVolumeSpecName: "kube-api-access-zzq97") pod "7e9f84bf-337f-401d-9e14-802a88a14e20" (UID: "7e9f84bf-337f-401d-9e14-802a88a14e20"). InnerVolumeSpecName "kube-api-access-zzq97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:42:56.920482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.920396    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzq97\" (UniqueName: \"kubernetes.io/projected/7e9f84bf-337f-401d-9e14-802a88a14e20-kube-api-access-zzq97\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:42:56.942168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.942129    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:56.942466 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.942453    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9f84bf-337f-401d-9e14-802a88a14e20" containerName="test-curl-7255-139727531079216"
Jul 30 01:42:56.942514 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.942467    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9f84bf-337f-401d-9e14-802a88a14e20" containerName="test-curl-7255-139727531079216"
Jul 30 01:42:56.942550 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.942534    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e9f84bf-337f-401d-9e14-802a88a14e20" containerName="test-curl-7255-139727531079216"
Jul 30 01:42:56.944456 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.944440    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:56.951152 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:56.951118    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:57.021168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.021133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvm4h\" (UniqueName: \"kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h\") pod \"test-curl-7255-139727531043984\" (UID: \"6995cdbd-d9de-4b10-a6d5-164ed8cf5878\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:57.122589 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.122552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvm4h\" (UniqueName: \"kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h\") pod \"test-curl-7255-139727531043984\" (UID: \"6995cdbd-d9de-4b10-a6d5-164ed8cf5878\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:57.130683 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.130658    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvm4h\" (UniqueName: \"kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h\") pod \"test-curl-7255-139727531043984\" (UID: \"6995cdbd-d9de-4b10-a6d5-164ed8cf5878\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:57.254143 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.254121    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:57.386394 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.386361    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e9f84bf-337f-401d-9e14-802a88a14e20" path="/var/lib/kubelet/pods/7e9f84bf-337f-401d-9e14-802a88a14e20/volumes"
Jul 30 01:42:57.386830 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.386778    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:57.386938 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:42:57.386860    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6995cdbd_d9de_4b10_a6d5_164ed8cf5878.slice/crio-b1b4a5dbc6a484cad0700f52d57b29d95c169d5cef418bc2ef2bc16aa8c6e208 WatchSource:0}: Error finding container b1b4a5dbc6a484cad0700f52d57b29d95c169d5cef418bc2ef2bc16aa8c6e208: Status 404 returned error can't find the container with id b1b4a5dbc6a484cad0700f52d57b29d95c169d5cef418bc2ef2bc16aa8c6e208
Jul 30 01:42:57.619988 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.619910    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531043984" event={"ID":"6995cdbd-d9de-4b10-a6d5-164ed8cf5878","Type":"ContainerStarted","Data":"b1b4a5dbc6a484cad0700f52d57b29d95c169d5cef418bc2ef2bc16aa8c6e208"}
Jul 30 01:42:57.621076 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.621052    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531079216"
Jul 30 01:42:57.621206 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:57.621103    2584 scope.go:117] "RemoveContainer" containerID="32c2c7dc632bd030b572d018dd5a2f9a81b47d10c059fc4cf27cb95598aa0fd3"
Jul 30 01:42:58.627910 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:58.627880    2584 generic.go:356] "Generic (PLEG): container finished" podID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" containerID="c51aab9b5d63abf6a07cbc87d1be78f9c251af65870b7a27a75ef20abe63c876" exitCode=0
Jul 30 01:42:58.628332 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:58.627953    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531043984" event={"ID":"6995cdbd-d9de-4b10-a6d5-164ed8cf5878","Type":"ContainerDied","Data":"c51aab9b5d63abf6a07cbc87d1be78f9c251af65870b7a27a75ef20abe63c876"}
Jul 30 01:42:59.772073 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.772050    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:59.791179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.791150    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531043984_6995cdbd-d9de-4b10-a6d5-164ed8cf5878/test-curl-7255-139727531043984/0.log"
Jul 30 01:42:59.811852 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.811827    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:59.815531 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.815507    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:59.846753 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.846729    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvm4h\" (UniqueName: \"kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h\") pod \"6995cdbd-d9de-4b10-a6d5-164ed8cf5878\" (UID: \"6995cdbd-d9de-4b10-a6d5-164ed8cf5878\") "
Jul 30 01:42:59.848734 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.848705    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h" (OuterVolumeSpecName: "kube-api-access-lvm4h") pod "6995cdbd-d9de-4b10-a6d5-164ed8cf5878" (UID: "6995cdbd-d9de-4b10-a6d5-164ed8cf5878"). InnerVolumeSpecName "kube-api-access-lvm4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:42:59.944362 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.944282    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:42:59.944648 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.944635    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" containerName="test-curl-7255-139727531043984"
Jul 30 01:42:59.944692 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.944649    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" containerName="test-curl-7255-139727531043984"
Jul 30 01:42:59.944730 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.944717    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" containerName="test-curl-7255-139727531043984"
Jul 30 01:42:59.946641 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.946624    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:42:59.948527 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.948502    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvm4h\" (UniqueName: \"kubernetes.io/projected/6995cdbd-d9de-4b10-a6d5-164ed8cf5878-kube-api-access-lvm4h\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:42:59.953572 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:42:59.953550    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:43:00.049519 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.049486    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvqnn\" (UniqueName: \"kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn\") pod \"test-curl-7255-139727531043984\" (UID: \"0e30da81-f2e2-4fae-b7bc-a5c0b576331e\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:00.150999 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.150961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvqnn\" (UniqueName: \"kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn\") pod \"test-curl-7255-139727531043984\" (UID: \"0e30da81-f2e2-4fae-b7bc-a5c0b576331e\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:00.159200 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.159171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvqnn\" (UniqueName: \"kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn\") pod \"test-curl-7255-139727531043984\" (UID: \"0e30da81-f2e2-4fae-b7bc-a5c0b576331e\") " pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:00.256158 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.256127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:00.381095 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.381071    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:43:00.382990 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:43:00.382960    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e30da81_f2e2_4fae_b7bc_a5c0b576331e.slice/crio-c5e13cfe831ee3ce6f0e0365e1a3e67daea62d61faa4e0d72e02de6d2552dd7a WatchSource:0}: Error finding container c5e13cfe831ee3ce6f0e0365e1a3e67daea62d61faa4e0d72e02de6d2552dd7a: Status 404 returned error can't find the container with id c5e13cfe831ee3ce6f0e0365e1a3e67daea62d61faa4e0d72e02de6d2552dd7a
Jul 30 01:43:00.646216 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.646129    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531043984" event={"ID":"0e30da81-f2e2-4fae-b7bc-a5c0b576331e","Type":"ContainerStarted","Data":"c5e13cfe831ee3ce6f0e0365e1a3e67daea62d61faa4e0d72e02de6d2552dd7a"}
Jul 30 01:43:00.647334 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.647315    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:00.647472 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.647317    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1b4a5dbc6a484cad0700f52d57b29d95c169d5cef418bc2ef2bc16aa8c6e208"
Jul 30 01:43:00.660638 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:00.660606    2584 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7255-139727531043984" oldPodUID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" podUID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e"
Jul 30 01:43:01.385334 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:01.385301    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6995cdbd-d9de-4b10-a6d5-164ed8cf5878" path="/var/lib/kubelet/pods/6995cdbd-d9de-4b10-a6d5-164ed8cf5878/volumes"
Jul 30 01:43:01.653823 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:01.653727    2584 generic.go:356] "Generic (PLEG): container finished" podID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e" containerID="75e6e91fa55d3a602e15b6b7778353e701bbb060a3a67220eac6d07267eb4ab8" exitCode=0
Jul 30 01:43:01.653985 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:01.653825    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531043984" event={"ID":"0e30da81-f2e2-4fae-b7bc-a5c0b576331e","Type":"ContainerDied","Data":"75e6e91fa55d3a602e15b6b7778353e701bbb060a3a67220eac6d07267eb4ab8"}
Jul 30 01:43:01.838158 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:01.838125    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:43:02.791423 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.791398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:02.811452 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.811426    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531043984_0e30da81-f2e2-4fae-b7bc-a5c0b576331e/test-curl-7255-139727531043984/0.log"
Jul 30 01:43:02.831452 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.831426    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:43:02.836907 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.836883    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531043984"]
Jul 30 01:43:02.875404 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.875380    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvqnn\" (UniqueName: \"kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn\") pod \"0e30da81-f2e2-4fae-b7bc-a5c0b576331e\" (UID: \"0e30da81-f2e2-4fae-b7bc-a5c0b576331e\") "
Jul 30 01:43:02.877773 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.877741    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn" (OuterVolumeSpecName: "kube-api-access-dvqnn") pod "0e30da81-f2e2-4fae-b7bc-a5c0b576331e" (UID: "0e30da81-f2e2-4fae-b7bc-a5c0b576331e"). InnerVolumeSpecName "kube-api-access-dvqnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:43:02.963132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.963053    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:02.963398 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.963386    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e" containerName="test-curl-7255-139727531043984"
Jul 30 01:43:02.963447 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.963399    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e" containerName="test-curl-7255-139727531043984"
Jul 30 01:43:02.963482 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.963456    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e" containerName="test-curl-7255-139727531043984"
Jul 30 01:43:02.966020 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.966003    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:02.972056 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.972030    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:02.976037 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:02.976010    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvqnn\" (UniqueName: \"kubernetes.io/projected/0e30da81-f2e2-4fae-b7bc-a5c0b576331e-kube-api-access-dvqnn\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:03.076576 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.076528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssm5j\" (UniqueName: \"kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j\") pod \"test-curl-7255-139727531171888\" (UID: \"1aa72398-78ae-440c-9bbf-a00ad42a2fbd\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:03.177317 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.177282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssm5j\" (UniqueName: \"kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j\") pod \"test-curl-7255-139727531171888\" (UID: \"1aa72398-78ae-440c-9bbf-a00ad42a2fbd\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:03.185483 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.185451    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssm5j\" (UniqueName: \"kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j\") pod \"test-curl-7255-139727531171888\" (UID: \"1aa72398-78ae-440c-9bbf-a00ad42a2fbd\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:03.276432 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.276398    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:03.388952 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.388911    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e30da81-f2e2-4fae-b7bc-a5c0b576331e" path="/var/lib/kubelet/pods/0e30da81-f2e2-4fae-b7bc-a5c0b576331e/volumes"
Jul 30 01:43:03.615198 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.615174    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:03.617242 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:43:03.617219    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1aa72398_78ae_440c_9bbf_a00ad42a2fbd.slice/crio-452acae6488bc2c0d91081aaef3744b5d11ef1af3aebda236e1c43be5f1a7dc4 WatchSource:0}: Error finding container 452acae6488bc2c0d91081aaef3744b5d11ef1af3aebda236e1c43be5f1a7dc4: Status 404 returned error can't find the container with id 452acae6488bc2c0d91081aaef3744b5d11ef1af3aebda236e1c43be5f1a7dc4
Jul 30 01:43:03.664980 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.664933    2584 scope.go:117] "RemoveContainer" containerID="75e6e91fa55d3a602e15b6b7778353e701bbb060a3a67220eac6d07267eb4ab8"
Jul 30 01:43:03.665088 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.664961    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531043984"
Jul 30 01:43:03.666170 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.666145    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531171888" event={"ID":"1aa72398-78ae-440c-9bbf-a00ad42a2fbd","Type":"ContainerStarted","Data":"452acae6488bc2c0d91081aaef3744b5d11ef1af3aebda236e1c43be5f1a7dc4"}
Jul 30 01:43:03.765194 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:03.765149    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" podUID="747214a1-7902-45c3-9ebd-1f6d9019aa49" containerName="main" containerID="cri-o://09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532" gracePeriod=45
Jul 30 01:43:04.072514 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.072493    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:43:04.184786 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184716    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp4fq\" (UniqueName: \"kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.184786 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184760    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.185024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184811    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.185024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184865    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.185024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184890    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.185024 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.184917    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm\") pod \"747214a1-7902-45c3-9ebd-1f6d9019aa49\" (UID: \"747214a1-7902-45c3-9ebd-1f6d9019aa49\") "
Jul 30 01:43:04.185210 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185047    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache" (OuterVolumeSpecName: "model-cache") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:04.185210 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185133    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home" (OuterVolumeSpecName: "home") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:04.185210 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185137    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:04.185348 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185271    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.185348 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185288    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.185348 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.185297    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.186907 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.186873    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq" (OuterVolumeSpecName: "kube-api-access-wp4fq") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "kube-api-access-wp4fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:43:04.187015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.186991    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:43:04.187015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.187001    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm" (OuterVolumeSpecName: "dshm") pod "747214a1-7902-45c3-9ebd-1f6d9019aa49" (UID: "747214a1-7902-45c3-9ebd-1f6d9019aa49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:04.286563 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.286511    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp4fq\" (UniqueName: \"kubernetes.io/projected/747214a1-7902-45c3-9ebd-1f6d9019aa49-kube-api-access-wp4fq\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.286563 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.286557    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747214a1-7902-45c3-9ebd-1f6d9019aa49-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.286563 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.286568    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747214a1-7902-45c3-9ebd-1f6d9019aa49-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:04.671723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.671691    2584 generic.go:356] "Generic (PLEG): container finished" podID="747214a1-7902-45c3-9ebd-1f6d9019aa49" containerID="09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532" exitCode=0
Jul 30 01:43:04.671931 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.671733    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"
Jul 30 01:43:04.671931 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.671778    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" event={"ID":"747214a1-7902-45c3-9ebd-1f6d9019aa49","Type":"ContainerDied","Data":"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"}
Jul 30 01:43:04.671931 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.671838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg" event={"ID":"747214a1-7902-45c3-9ebd-1f6d9019aa49","Type":"ContainerDied","Data":"c1e215ab734197a8596751400c475a829be8d192709be7d17f40c676f2d977dc"}
Jul 30 01:43:04.671931 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.671861    2584 scope.go:117] "RemoveContainer" containerID="09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"
Jul 30 01:43:04.673332 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.673311    2584 generic.go:356] "Generic (PLEG): container finished" podID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" containerID="bf47ef81c5b82d778ec654861c87c55ff7e08f68c7ae16f5769bbeb8fad1005e" exitCode=0
Jul 30 01:43:04.673414 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.673341    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531171888" event={"ID":"1aa72398-78ae-440c-9bbf-a00ad42a2fbd","Type":"ContainerDied","Data":"bf47ef81c5b82d778ec654861c87c55ff7e08f68c7ae16f5769bbeb8fad1005e"}
Jul 30 01:43:04.681214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.681099    2584 scope.go:117] "RemoveContainer" containerID="09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"
Jul 30 01:43:04.681412 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:43:04.681389    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532\": container with ID starting with 09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532 not found: ID does not exist" containerID="09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"
Jul 30 01:43:04.681485 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.681421    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532"} err="failed to get container status \"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532\": rpc error: code = NotFound desc = could not find container \"09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532\": container with ID starting with 09d43066ac33d6e8e7002ec012a18e66df9517617d138715684bec6245489532 not found: ID does not exist"
Jul 30 01:43:04.698143 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.698124    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:43:04.700398 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.700375    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-55b321/rate-test-model-55b321-kserve-6fc47864b7-lz2sg"]
Jul 30 01:43:04.948940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:04.948848    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" podUID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" containerName="main" containerID="cri-o://af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6" gracePeriod=45
Jul 30 01:43:05.197073 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.197049    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:43:05.296681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296649    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.296884 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296706    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.296884 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296751    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ngh6\" (UniqueName: \"kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.296884 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296778    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.296884 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296871    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.297106 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.296946    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:05.297106 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297018    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir\") pod \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\" (UID: \"ff3882a4-3b77-4d18-8bb9-72bbacd97e43\") "
Jul 30 01:43:05.297204 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297099    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home" (OuterVolumeSpecName: "home") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:05.297261 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297222    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:05.297412 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297393    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.297490 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297414    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.297490 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.297426    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.298924 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.298898    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm" (OuterVolumeSpecName: "dshm") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:43:05.299034 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.298936    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:43:05.299212 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.299193    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6" (OuterVolumeSpecName: "kube-api-access-7ngh6") pod "ff3882a4-3b77-4d18-8bb9-72bbacd97e43" (UID: "ff3882a4-3b77-4d18-8bb9-72bbacd97e43"). InnerVolumeSpecName "kube-api-access-7ngh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:43:05.385440 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.385408    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="747214a1-7902-45c3-9ebd-1f6d9019aa49" path="/var/lib/kubelet/pods/747214a1-7902-45c3-9ebd-1f6d9019aa49/volumes"
Jul 30 01:43:05.398723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.398701    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.398848 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.398726    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ngh6\" (UniqueName: \"kubernetes.io/projected/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-kube-api-access-7ngh6\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.398848 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.398735    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3882a4-3b77-4d18-8bb9-72bbacd97e43-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:05.681241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.681152    2584 generic.go:356] "Generic (PLEG): container finished" podID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" containerID="af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6" exitCode=0
Jul 30 01:43:05.681241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.681199    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"
Jul 30 01:43:05.681431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.681241    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" event={"ID":"ff3882a4-3b77-4d18-8bb9-72bbacd97e43","Type":"ContainerDied","Data":"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"}
Jul 30 01:43:05.681431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.681290    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks" event={"ID":"ff3882a4-3b77-4d18-8bb9-72bbacd97e43","Type":"ContainerDied","Data":"542c508b55fb433e4126a0729e90f3532e43232e4065b4966f867d3f7b790a43"}
Jul 30 01:43:05.681431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.681321    2584 scope.go:117] "RemoveContainer" containerID="af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"
Jul 30 01:43:05.690869 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.690846    2584 scope.go:117] "RemoveContainer" containerID="af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"
Jul 30 01:43:05.691148 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:43:05.691129    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6\": container with ID starting with af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6 not found: ID does not exist" containerID="af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"
Jul 30 01:43:05.691217 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.691159    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6"} err="failed to get container status \"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6\": rpc error: code = NotFound desc = could not find container \"af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6\": container with ID starting with af8f45e492d901ea0be0597220ee12ab31b8fe7f5af5e0b0a9d6378747013be6 not found: ID does not exist"
Jul 30 01:43:05.696545 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.696517    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:43:05.700625 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.700601    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fba93c/rate-test-model-fba93c-kserve-6c9b54bc75-z9dks"]
Jul 30 01:43:05.817742 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.817718    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:05.836109 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.836083    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531171888_1aa72398-78ae-440c-9bbf-a00ad42a2fbd/test-curl-7255-139727531171888/0.log"
Jul 30 01:43:05.856755 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.856727    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:05.862834 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.862810    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:05.904483 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.904455    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssm5j\" (UniqueName: \"kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j\") pod \"1aa72398-78ae-440c-9bbf-a00ad42a2fbd\" (UID: \"1aa72398-78ae-440c-9bbf-a00ad42a2fbd\") "
Jul 30 01:43:05.906473 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.906453    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j" (OuterVolumeSpecName: "kube-api-access-ssm5j") pod "1aa72398-78ae-440c-9bbf-a00ad42a2fbd" (UID: "1aa72398-78ae-440c-9bbf-a00ad42a2fbd"). InnerVolumeSpecName "kube-api-access-ssm5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:43:05.987103 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987068    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:05.987415 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987403    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:05.987465 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987416    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:05.987465 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987439    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747214a1-7902-45c3-9ebd-1f6d9019aa49" containerName="main"
Jul 30 01:43:05.987465 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987445    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="747214a1-7902-45c3-9ebd-1f6d9019aa49" containerName="main"
Jul 30 01:43:05.987465 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987454    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" containerName="main"
Jul 30 01:43:05.987465 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987460    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" containerName="main"
Jul 30 01:43:05.987638 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987510    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="747214a1-7902-45c3-9ebd-1f6d9019aa49" containerName="main"
Jul 30 01:43:05.987638 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987517    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" containerName="main"
Jul 30 01:43:05.987638 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.987524    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:05.990439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.990423    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:05.998183 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:05.997936    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:06.005273 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.005249    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssm5j\" (UniqueName: \"kubernetes.io/projected/1aa72398-78ae-440c-9bbf-a00ad42a2fbd-kube-api-access-ssm5j\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:06.106532 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.106501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcfgv\" (UniqueName: \"kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv\") pod \"test-curl-7255-139727531171888\" (UID: \"85f8684e-7726-4c5a-9aa5-47ef364085c1\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:06.207559 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.207515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcfgv\" (UniqueName: \"kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv\") pod \"test-curl-7255-139727531171888\" (UID: \"85f8684e-7726-4c5a-9aa5-47ef364085c1\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:06.215523 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.215503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcfgv\" (UniqueName: \"kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv\") pod \"test-curl-7255-139727531171888\" (UID: \"85f8684e-7726-4c5a-9aa5-47ef364085c1\") " pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:06.301332 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.301245    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:06.428231 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.428205    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:06.429553 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:43:06.429522    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85f8684e_7726_4c5a_9aa5_47ef364085c1.slice/crio-df1f6ed1a9abef3367f587b0646effc3738be85d9536a2b362b50e6abbbeadad WatchSource:0}: Error finding container df1f6ed1a9abef3367f587b0646effc3738be85d9536a2b362b50e6abbbeadad: Status 404 returned error can't find the container with id df1f6ed1a9abef3367f587b0646effc3738be85d9536a2b362b50e6abbbeadad
Jul 30 01:43:06.687455 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.687375    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531171888" event={"ID":"85f8684e-7726-4c5a-9aa5-47ef364085c1","Type":"ContainerStarted","Data":"df1f6ed1a9abef3367f587b0646effc3738be85d9536a2b362b50e6abbbeadad"}
Jul 30 01:43:06.689419 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.689399    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="452acae6488bc2c0d91081aaef3744b5d11ef1af3aebda236e1c43be5f1a7dc4"
Jul 30 01:43:06.689544 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.689426    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:06.700385 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:06.700360    2584 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7255-139727531171888" oldPodUID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" podUID="85f8684e-7726-4c5a-9aa5-47ef364085c1"
Jul 30 01:43:07.385603 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:07.385566    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aa72398-78ae-440c-9bbf-a00ad42a2fbd" path="/var/lib/kubelet/pods/1aa72398-78ae-440c-9bbf-a00ad42a2fbd/volumes"
Jul 30 01:43:07.385995 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:07.385983    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3882a4-3b77-4d18-8bb9-72bbacd97e43" path="/var/lib/kubelet/pods/ff3882a4-3b77-4d18-8bb9-72bbacd97e43/volumes"
Jul 30 01:43:07.696131 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:07.696041    2584 generic.go:356] "Generic (PLEG): container finished" podID="85f8684e-7726-4c5a-9aa5-47ef364085c1" containerID="30ea5342dbfd4b19efced459280fe70bed9e2e447322404e781f15bf0f297966" exitCode=0
Jul 30 01:43:07.696261 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:07.696133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7255-139727531171888" event={"ID":"85f8684e-7726-4c5a-9aa5-47ef364085c1","Type":"ContainerDied","Data":"30ea5342dbfd4b19efced459280fe70bed9e2e447322404e781f15bf0f297966"}
Jul 30 01:43:08.830855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.830830    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:08.849663 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.849613    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7255-139727531171888_85f8684e-7726-4c5a-9aa5-47ef364085c1/test-curl-7255-139727531171888/0.log"
Jul 30 01:43:08.869135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.869100    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:08.872508 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.872485    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7255-139727531171888"]
Jul 30 01:43:08.931045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.931005    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcfgv\" (UniqueName: \"kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv\") pod \"85f8684e-7726-4c5a-9aa5-47ef364085c1\" (UID: \"85f8684e-7726-4c5a-9aa5-47ef364085c1\") "
Jul 30 01:43:08.933104 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:08.933070    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv" (OuterVolumeSpecName: "kube-api-access-gcfgv") pod "85f8684e-7726-4c5a-9aa5-47ef364085c1" (UID: "85f8684e-7726-4c5a-9aa5-47ef364085c1"). InnerVolumeSpecName "kube-api-access-gcfgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:43:09.031810 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.031752    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcfgv\" (UniqueName: \"kubernetes.io/projected/85f8684e-7726-4c5a-9aa5-47ef364085c1-kube-api-access-gcfgv\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:43:09.385744 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.385649    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f8684e-7726-4c5a-9aa5-47ef364085c1" path="/var/lib/kubelet/pods/85f8684e-7726-4c5a-9aa5-47ef364085c1/volumes"
Jul 30 01:43:09.631975 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.631940    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:43:09.632298 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.632283    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f8684e-7726-4c5a-9aa5-47ef364085c1" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:09.632356 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.632299    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f8684e-7726-4c5a-9aa5-47ef364085c1" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:09.632409 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.632397    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f8684e-7726-4c5a-9aa5-47ef364085c1" containerName="test-curl-7255-139727531171888"
Jul 30 01:43:09.635538 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.635522    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.637918 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.637864    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-09b0d2-openshift-default-dockercfg-l659m\""
Jul 30 01:43:09.648391 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.648364    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:43:09.706337 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.706315    2584 scope.go:117] "RemoveContainer" containerID="30ea5342dbfd4b19efced459280fe70bed9e2e447322404e781f15bf0f297966"
Jul 30 01:43:09.706516 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.706315    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7255-139727531171888"
Jul 30 01:43:09.737388 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737360    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnwg\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737505 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737422    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737575 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737498    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737575 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737689 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737616    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737689 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737649    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737769 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737688    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737769 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.737885 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.737815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839046 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839012    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnwg\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839120    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839157    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839188    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839459 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839905 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839626    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839905 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.839905 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.839733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.840069 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.840049    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.840130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.840106    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.841463 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.841435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.841724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.841704    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.846327 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.846307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.846578 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.846560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnwg\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg\") pod \"e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") " pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:09.945829 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:09.945716    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:10.079682 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.079643    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:43:10.080771 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:43:10.080740    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be293cf_7440_4744_9a4d_16f568b49db5.slice/crio-4b9d359041d69419b8c59dbd1b219a6d82f108c2516d6914db1a8e13ef531e45 WatchSource:0}: Error finding container 4b9d359041d69419b8c59dbd1b219a6d82f108c2516d6914db1a8e13ef531e45: Status 404 returned error can't find the container with id 4b9d359041d69419b8c59dbd1b219a6d82f108c2516d6914db1a8e13ef531e45
Jul 30 01:43:10.083146 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.083103    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:10.083241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.083195    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:10.083241 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.083236    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:10.714026 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.713994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerStarted","Data":"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"}
Jul 30 01:43:10.714177 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.714043    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerStarted","Data":"4b9d359041d69419b8c59dbd1b219a6d82f108c2516d6914db1a8e13ef531e45"}
Jul 30 01:43:10.734319 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.734269    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podStartSLOduration=1.734254577 podStartE2EDuration="1.734254577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:10.731684808 +0000 UTC m=+3577.922633139" watchObservedRunningTime="2026-07-30 01:43:10.734254577 +0000 UTC m=+3577.925202908"
Jul 30 01:43:10.946430 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:10.946395    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:11.950552 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:11.950517    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:12.722704 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:12.722671    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:12.723574 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:12.723556    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:19.237923 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:19.237887    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:43:27.724308 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.724271    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 30 01:43:27.724722 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.724334    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 30 01:43:27.794985 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.794954    2584 generic.go:356] "Generic (PLEG): container finished" podID="2be293cf-7440-4744-9a4d-16f568b49db5" containerID="dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642" exitCode=0
Jul 30 01:43:27.795145 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.795026    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerDied","Data":"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"}
Jul 30 01:43:27.795411 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.795397    2584 scope.go:117] "RemoveContainer" containerID="dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"
Jul 30 01:43:27.796385 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.796361    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:27.796442 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.796429    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:27.796481 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:27.796462    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:28.801874 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:28.801841    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerStarted","Data":"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"}
Jul 30 01:43:28.945886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:28.945847    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:28.946108 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:28.946085    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 30 01:43:28.946157 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:28.946132    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 30 01:43:29.946712 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:29.946673    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 30 01:43:29.947125 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:29.946761    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 30 01:43:30.946962 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:30.946928    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 30 01:43:30.947330 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:30.947006    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 30 01:43:31.946233 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:31.946194    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 30 01:43:31.946430 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:31.946251    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 30 01:43:32.950147 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:32.950113    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:32.950571 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:32.950356    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:32.951025 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:32.951005    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:43:38.550016 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.549981    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:43:38.554992 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.554970    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.557423 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.557403    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-b64c39-openshift-default-dockercfg-mkt47\""
Jul 30 01:43:38.565818 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.565776    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:43:38.589521 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589492    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589652 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589529    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589652 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589554    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2fw6\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589639    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589842 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589844    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589882 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.589958 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.589898    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690661 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690679    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690743    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690762    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.690886 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690907    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2fw6\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.690951    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.691103    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691214 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.691168    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.691273    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691431 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.691325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.691828 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.691782    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.693154 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.693131    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.693529 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.693510    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.698622 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.698602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.698726 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.698706    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2fw6\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6\") pod \"e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") " pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.866423 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.866339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:38.998880 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:38.998850    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:43:39.000682 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:43:39.000654    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de1a192_5a5f_4424_958a_f05f991821fa.slice/crio-a54671fdfb2d4a006edff327ae67ae7f8fce38ec7284d1b6a913d506a5c558c0 WatchSource:0}: Error finding container a54671fdfb2d4a006edff327ae67ae7f8fce38ec7284d1b6a913d506a5c558c0: Status 404 returned error can't find the container with id a54671fdfb2d4a006edff327ae67ae7f8fce38ec7284d1b6a913d506a5c558c0
Jul 30 01:43:39.003121 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.003087    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:39.003197 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.003173    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:39.003233 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.003214    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:39.862285 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.862242    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerStarted","Data":"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"}
Jul 30 01:43:39.862723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.862299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerStarted","Data":"a54671fdfb2d4a006edff327ae67ae7f8fce38ec7284d1b6a913d506a5c558c0"}
Jul 30 01:43:39.866468 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.866450    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:39.884180 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:39.884116    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podStartSLOduration=1.884103399 podStartE2EDuration="1.884103399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:39.882432735 +0000 UTC m=+3607.073381067" watchObservedRunningTime="2026-07-30 01:43:39.884103399 +0000 UTC m=+3607.075051741"
Jul 30 01:43:40.871925 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:40.871891    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:41.871349 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:41.871309    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:41.872327 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:41.872300    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:43:51.636950 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:51.636917    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:43:57.912969 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:57.912926    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 01:43:57.913376 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:57.912978    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 01:43:59.970480 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.970449    2584 generic.go:356] "Generic (PLEG): container finished" podID="8de1a192-5a5f-4424-958a-f05f991821fa" containerID="1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a" exitCode=0
Jul 30 01:43:59.971005 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.970521    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerDied","Data":"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"}
Jul 30 01:43:59.971080 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.971042    2584 scope.go:117] "RemoveContainer" containerID="1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"
Jul 30 01:43:59.972110 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.972084    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:59.972179 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.972151    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:43:59.972218 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:43:59.972182    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 01:44:00.977587 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:00.977550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerStarted","Data":"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"}
Jul 30 01:44:01.867441 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:01.867402    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:44:01.867669 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:01.867646    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 01:44:01.867751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:01.867725    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 01:44:02.866861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:02.866824    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 01:44:02.867256 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:02.866895    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 01:44:03.867171 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:03.867139    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 01:44:03.867566 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:03.867195    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 01:44:04.867201 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:04.867166    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 01:44:04.867592 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:04.867221    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 01:44:05.870533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:05.870501    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:44:05.870977 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:05.870761    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:44:05.871303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:05.871287    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:44:26.545970 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.545933    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:44:26.548287 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.548271    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.550598 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.550577    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-09b0d2\"/\"kube-root-ca.crt\""
Jul 30 01:44:26.551536 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.551516    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-09b0d2\"/\"default-dockercfg-tccpb\""
Jul 30 01:44:26.551667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.551561    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-09b0d2\"/\"openshift-service-ca.crt\""
Jul 30 01:44:26.551667 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.551589    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-09b0d2\"/\"ipp-route-09b0d2-kserve-self-signed-certs\""
Jul 30 01:44:26.557499 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.557477    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:44:26.628701 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628669    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.628701 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.628944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628733    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.628944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.628944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628856    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.628944 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.628925    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4bdt\" (UniqueName: \"kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729354 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729386    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4bdt\" (UniqueName: \"kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729415    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729441    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729808 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729859 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729817    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.729903 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.729855    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.731741 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.731721    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.731896 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.731879    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.737073 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.737053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4bdt\" (UniqueName: \"kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt\") pod \"ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") " pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.858427 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.858332    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:26.992405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:26.992376    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:44:26.994822 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:44:26.994783    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c168ffa_3f8f_43d7_b166_e032b3c85086.slice/crio-e06776604a3e7c6fd2140dcfba12164dcf0942934dabc7ae63fba674305307b5 WatchSource:0}: Error finding container e06776604a3e7c6fd2140dcfba12164dcf0942934dabc7ae63fba674305307b5: Status 404 returned error can't find the container with id e06776604a3e7c6fd2140dcfba12164dcf0942934dabc7ae63fba674305307b5
Jul 30 01:44:27.108144 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:27.108049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" event={"ID":"6c168ffa-3f8f-43d7-b166-e032b3c85086","Type":"ContainerStarted","Data":"17becc1f4460702641714c24dac8852dac65116156c41e2b91132606431f19b4"}
Jul 30 01:44:27.108144 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:27.108114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" event={"ID":"6c168ffa-3f8f-43d7-b166-e032b3c85086","Type":"ContainerStarted","Data":"e06776604a3e7c6fd2140dcfba12164dcf0942934dabc7ae63fba674305307b5"}
Jul 30 01:44:27.125901 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:27.125790    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" podStartSLOduration=1.125773816 podStartE2EDuration="1.125773816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:44:27.123193073 +0000 UTC m=+3654.314141404" watchObservedRunningTime="2026-07-30 01:44:27.125773816 +0000 UTC m=+3654.316722147"
Jul 30 01:44:30.640428 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:30.640391    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:44:36.858567 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:36.858524    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:36.858567 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:36.858576    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:36.871059 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:36.871032    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:44:37.172397 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:44:37.172301    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:45:04.471366 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:04.471324    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"]
Jul 30 01:45:04.477551 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:04.477522    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756220-k6n69"]
Jul 30 01:45:05.385676 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:05.385641    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46bf7bad-0b78-4799-8786-58e46c0e3927" path="/var/lib/kubelet/pods/46bf7bad-0b78-4799-8786-58e46c0e3927/volumes"
Jul 30 01:45:10.736647 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:10.736610    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:45:30.238954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:30.238918    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:45:33.803814 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:33.803761    2584 scope.go:117] "RemoveContainer" containerID="334baabf433d2bf199eef626a9b4f0d2feecc0802e5d9c07bdbdf1ddede04622"
Jul 30 01:45:50.223396 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.223357    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"]
Jul 30 01:45:50.226626 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.226608    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:50.229055 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.229033    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:45:50.229212 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.229192    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:45:50.235058 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.235038    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"]
Jul 30 01:45:50.308775 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.308745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr\" (UID: \"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:50.409660 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.409621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr\" (UID: \"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:50.410302 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.410277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr\" (UID: \"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:50.538993 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.538944    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:50.673341 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:50.673106    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"]
Jul 30 01:45:50.675540 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:45:50.675509    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b5eff9_4ef2_4ec5_8d8b_54bd09c756e3.slice/crio-e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762 WatchSource:0}: Error finding container e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762: Status 404 returned error can't find the container with id e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762
Jul 30 01:45:51.522444 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:51.522410    2584 generic.go:356] "Generic (PLEG): container finished" podID="49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" containerID="22295781852ab23ff59e32715167f3aaadc5d87fbfa509022f8bed82ea85f2d8" exitCode=0
Jul 30 01:45:51.522833 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:51.522485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr" event={"ID":"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3","Type":"ContainerDied","Data":"22295781852ab23ff59e32715167f3aaadc5d87fbfa509022f8bed82ea85f2d8"}
Jul 30 01:45:51.522833 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:51.522526    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr" event={"ID":"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3","Type":"ContainerStarted","Data":"e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762"}
Jul 30 01:45:52.657581 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:52.657559    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:52.731339 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:52.731305    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca\") pod \"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3\" (UID: \"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3\") "
Jul 30 01:45:52.731685 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:52.731661    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" (UID: "49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:45:52.831901 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:52.831836    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:45:53.533135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:53.533102    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr" event={"ID":"49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3","Type":"ContainerDied","Data":"e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762"}
Jul 30 01:45:53.533135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:53.533120    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"
Jul 30 01:45:53.533337 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:53.533151    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6bc56cba0fb91cb2757b9548d318fa79e2ab6f8b4d7a182348bfe17d3cb8762"
Jul 30 01:45:56.438568 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:56.438530    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"]
Jul 30 01:45:56.444974 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:56.444947    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-06c66b-mtvwr"]
Jul 30 01:45:57.387001 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:57.386967    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" path="/var/lib/kubelet/pods/49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3/volumes"
Jul 30 01:45:59.144590 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:45:59.144556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:46:06.489451 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.489418    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"]
Jul 30 01:46:06.489940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.489766    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" containerName="revoke-keys"
Jul 30 01:46:06.489940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.489776    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" containerName="revoke-keys"
Jul 30 01:46:06.489940 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.489869    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b5eff9-4ef2-4ec5-8d8b-54bd09c756e3" containerName="revoke-keys"
Jul 30 01:46:06.492984 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.492967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:06.495434 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.495406    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:46:06.495564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.495481    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:46:06.500741 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.500716    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"]
Jul 30 01:46:06.552583 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.552552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr\" (UID: \"e5584889-b28e-4e6e-b16c-c2923c6a3e3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:06.653624 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.653583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr\" (UID: \"e5584889-b28e-4e6e-b16c-c2923c6a3e3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:06.654271 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.654247    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr\" (UID: \"e5584889-b28e-4e6e-b16c-c2923c6a3e3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:06.804702 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.804620    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:06.934725 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:06.934696    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"]
Jul 30 01:46:06.936747 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:46:06.936716    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5584889_b28e_4e6e_b16c_c2923c6a3e3a.slice/crio-bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319 WatchSource:0}: Error finding container bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319: Status 404 returned error can't find the container with id bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319
Jul 30 01:46:07.606956 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:07.606919    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr" event={"ID":"e5584889-b28e-4e6e-b16c-c2923c6a3e3a","Type":"ContainerStarted","Data":"22e5f0e51b49ac568980d223435855c8b115e604602541d0741b4d3be72cb109"}
Jul 30 01:46:07.607345 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:07.606976    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr" event={"ID":"e5584889-b28e-4e6e-b16c-c2923c6a3e3a","Type":"ContainerStarted","Data":"bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319"}
Jul 30 01:46:07.624100 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:07.624057    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr" podStartSLOduration=1.624040753 podStartE2EDuration="1.624040753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:46:07.622320323 +0000 UTC m=+3754.813268654" watchObservedRunningTime="2026-07-30 01:46:07.624040753 +0000 UTC m=+3754.814989084"
Jul 30 01:46:08.613385 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:08.613344    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5584889-b28e-4e6e-b16c-c2923c6a3e3a" containerID="22e5f0e51b49ac568980d223435855c8b115e604602541d0741b4d3be72cb109" exitCode=0
Jul 30 01:46:08.613771 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:08.613429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr" event={"ID":"e5584889-b28e-4e6e-b16c-c2923c6a3e3a","Type":"ContainerDied","Data":"22e5f0e51b49ac568980d223435855c8b115e604602541d0741b4d3be72cb109"}
Jul 30 01:46:09.748724 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:09.748702    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:09.884987 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:09.884902    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca\") pod \"e5584889-b28e-4e6e-b16c-c2923c6a3e3a\" (UID: \"e5584889-b28e-4e6e-b16c-c2923c6a3e3a\") "
Jul 30 01:46:09.885268 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:09.885246    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5584889-b28e-4e6e-b16c-c2923c6a3e3a" (UID: "e5584889-b28e-4e6e-b16c-c2923c6a3e3a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:46:09.986119 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:09.986091    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5584889-b28e-4e6e-b16c-c2923c6a3e3a-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:10.627395 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:10.627365    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"
Jul 30 01:46:10.627562 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:10.627364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr" event={"ID":"e5584889-b28e-4e6e-b16c-c2923c6a3e3a","Type":"ContainerDied","Data":"bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319"}
Jul 30 01:46:10.627562 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:10.627487    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb2ecc0c1d296b4bab15d6f24aa57d825ecd614e02b4f1e409a2b375cbbbf319"
Jul 30 01:46:22.240488 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:22.240453    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"]
Jul 30 01:46:22.242313 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:22.242285    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-09b0d2-v8mmr"]
Jul 30 01:46:22.446340 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:22.446299    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:46:23.386305 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:23.386270    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5584889-b28e-4e6e-b16c-c2923c6a3e3a" path="/var/lib/kubelet/pods/e5584889-b28e-4e6e-b16c-c2923c6a3e3a/volumes"
Jul 30 01:46:27.599505 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:27.599423    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:46:33.043089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:33.043056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:46:42.618452 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.618410    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" podUID="6c168ffa-3f8f-43d7-b166-e032b3c85086" containerName="main" containerID="cri-o://17becc1f4460702641714c24dac8852dac65116156c41e2b91132606431f19b4" gracePeriod=45
Jul 30 01:46:42.802972 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.802929    2584 generic.go:356] "Generic (PLEG): container finished" podID="6c168ffa-3f8f-43d7-b166-e032b3c85086" containerID="17becc1f4460702641714c24dac8852dac65116156c41e2b91132606431f19b4" exitCode=0
Jul 30 01:46:42.803133 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.802990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" event={"ID":"6c168ffa-3f8f-43d7-b166-e032b3c85086","Type":"ContainerDied","Data":"17becc1f4460702641714c24dac8852dac65116156c41e2b91132606431f19b4"}
Jul 30 01:46:42.863520 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.863495    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:46:42.884607 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884538    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.884821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884628    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.884821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884655    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.884821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884702    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4bdt\" (UniqueName: \"kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.884821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884728    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.884821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884789    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home\") pod \"6c168ffa-3f8f-43d7-b166-e032b3c85086\" (UID: \"6c168ffa-3f8f-43d7-b166-e032b3c85086\") "
Jul 30 01:46:42.885130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.884987    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:46:42.885130 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.885020    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:46:42.885245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.885113    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home" (OuterVolumeSpecName: "home") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:46:42.885245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.885151    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-model-cache\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:42.885245 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.885184    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-tmp-dir\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:42.887362 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.887335    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt" (OuterVolumeSpecName: "kube-api-access-n4bdt") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "kube-api-access-n4bdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:46:42.887617 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.887586    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm" (OuterVolumeSpecName: "dshm") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:46:42.887734 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.887710    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c168ffa-3f8f-43d7-b166-e032b3c85086" (UID: "6c168ffa-3f8f-43d7-b166-e032b3c85086"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:46:42.985654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.985619    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-dshm\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:42.985654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.985649    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4bdt\" (UniqueName: \"kubernetes.io/projected/6c168ffa-3f8f-43d7-b166-e032b3c85086-kube-api-access-n4bdt\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:42.985654 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.985660    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c168ffa-3f8f-43d7-b166-e032b3c85086-home\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:42.985919 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:42.985668    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c168ffa-3f8f-43d7-b166-e032b3c85086-tls-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:46:43.808500 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:43.808469    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"
Jul 30 01:46:43.808954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:43.808465    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8" event={"ID":"6c168ffa-3f8f-43d7-b166-e032b3c85086","Type":"ContainerDied","Data":"e06776604a3e7c6fd2140dcfba12164dcf0942934dabc7ae63fba674305307b5"}
Jul 30 01:46:43.808954 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:43.808604    2584 scope.go:117] "RemoveContainer" containerID="17becc1f4460702641714c24dac8852dac65116156c41e2b91132606431f19b4"
Jul 30 01:46:43.824589 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:43.824564    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:46:43.830237 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:43.830213    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-09b0d2/ipp-route-09b0d2-kserve-cb4fc7d95-cmrx8"]
Jul 30 01:46:45.386610 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:46:45.386577    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c168ffa-3f8f-43d7-b166-e032b3c85086" path="/var/lib/kubelet/pods/6c168ffa-3f8f-43d7-b166-e032b3c85086/volumes"
Jul 30 01:47:24.473021 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:24.472990    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:47:24.473517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:24.473220    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy" containerID="cri-o://dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2" gracePeriod=30
Jul 30 01:47:29.729855 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.729824    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:47:29.913475 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913436    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913475 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913475    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913513    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913574    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913592    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913631    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlnwg\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913661    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.913723 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913709    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy\") pod \"2be293cf-7440-4744-9a4d-16f568b49db5\" (UID: \"2be293cf-7440-4744-9a4d-16f568b49db5\") "
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913827    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913911    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913925    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.913998    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.914094    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.914113    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be293cf-7440-4744-9a4d-16f568b49db5-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:29.914135 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.914126    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:29.914439 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.914138    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:29.914909 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.914886    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data" (OuterVolumeSpecName: "istio-data") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:29.915979 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.915953    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:47:29.916111 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.915982    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token" (OuterVolumeSpecName: "istio-token") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:29.916280 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.916258    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg" (OuterVolumeSpecName: "kube-api-access-xlnwg") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "kube-api-access-xlnwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:29.916325 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:29.916298    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2be293cf-7440-4744-9a4d-16f568b49db5" (UID: "2be293cf-7440-4744-9a4d-16f568b49db5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:30.015436 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.015397    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlnwg\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-kube-api-access-xlnwg\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:30.015436 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.015432    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:30.015436 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.015443    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be293cf-7440-4744-9a4d-16f568b49db5-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:30.015677 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.015451    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be293cf-7440-4744-9a4d-16f568b49db5-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:30.015677 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.015460    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be293cf-7440-4744-9a4d-16f568b49db5-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:30.032945 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.032912    2584 generic.go:356] "Generic (PLEG): container finished" podID="2be293cf-7440-4744-9a4d-16f568b49db5" containerID="dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2" exitCode=0
Jul 30 01:47:30.033099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.032968    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"
Jul 30 01:47:30.033099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.032992    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerDied","Data":"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"}
Jul 30 01:47:30.033099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.033065    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26" event={"ID":"2be293cf-7440-4744-9a4d-16f568b49db5","Type":"ContainerDied","Data":"4b9d359041d69419b8c59dbd1b219a6d82f108c2516d6914db1a8e13ef531e45"}
Jul 30 01:47:30.033099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.033095    2584 scope.go:117] "RemoveContainer" containerID="dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"
Jul 30 01:47:30.042015 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.042000    2584 scope.go:117] "RemoveContainer" containerID="dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"
Jul 30 01:47:30.049811 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.049783    2584 scope.go:117] "RemoveContainer" containerID="dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"
Jul 30 01:47:30.050094 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:47:30.050071    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2\": container with ID starting with dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2 not found: ID does not exist" containerID="dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"
Jul 30 01:47:30.050154 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.050101    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2"} err="failed to get container status \"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2\": rpc error: code = NotFound desc = could not find container \"dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2\": container with ID starting with dc3cbc6d5e5cd7170c3b5ad1c6ba46a9237b4b2d407b0921eade614f747cffb2 not found: ID does not exist"
Jul 30 01:47:30.050154 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.050119    2584 scope.go:117] "RemoveContainer" containerID="dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"
Jul 30 01:47:30.050332 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:47:30.050316    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642\": container with ID starting with dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642 not found: ID does not exist" containerID="dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"
Jul 30 01:47:30.050373 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.050336    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642"} err="failed to get container status \"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642\": rpc error: code = NotFound desc = could not find container \"dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642\": container with ID starting with dd27f231d989dabbe820586212fbc0abdc486df1e6b803cd4cf1c5fd53702642 not found: ID does not exist"
Jul 30 01:47:30.053653 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.053630    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:47:30.056839 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:30.056815    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-09b0d2-openshift-default-584b576fdd-7rb26"]
Jul 30 01:47:31.385719 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:31.385690    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" path="/var/lib/kubelet/pods/2be293cf-7440-4744-9a4d-16f568b49db5/volumes"
Jul 30 01:47:38.350731 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:38.350690    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:47:41.668689 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:41.668648    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:47:41.669142 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:41.668913    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy" containerID="cri-o://2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0" gracePeriod=30
Jul 30 01:47:46.915782 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.915757    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:47:46.959490 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959417    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959490 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959453    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959502    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2fw6\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959547    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959576    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959600    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959639    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.959709 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959663    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.960042 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959723    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:46.960042 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959781    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token\") pod \"8de1a192-5a5f-4424-958a-f05f991821fa\" (UID: \"8de1a192-5a5f-4424-958a-f05f991821fa\") "
Jul 30 01:47:46.960042 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.959938    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:46.960184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960094    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:46.960184 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960102    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:46.960291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960229    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8de1a192-5a5f-4424-958a-f05f991821fa-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:46.960291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960246    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:46.960291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960259    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:46.960291 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960274    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:46.960493 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.960469    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:46.962031 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.961999    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:46.962140 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.962094    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6" (OuterVolumeSpecName: "kube-api-access-w2fw6") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "kube-api-access-w2fw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:46.962239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.962218    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:47:46.962287 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:46.962217    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8de1a192-5a5f-4424-958a-f05f991821fa" (UID: "8de1a192-5a5f-4424-958a-f05f991821fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:47:47.061005 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.060971    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:47.061005 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.061000    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2fw6\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-kube-api-access-w2fw6\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:47.061005 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.061010    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8de1a192-5a5f-4424-958a-f05f991821fa-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:47.061239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.061018    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8de1a192-5a5f-4424-958a-f05f991821fa-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:47.061239 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.061026    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8de1a192-5a5f-4424-958a-f05f991821fa-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:47:47.113506 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.113467    2584 generic.go:356] "Generic (PLEG): container finished" podID="8de1a192-5a5f-4424-958a-f05f991821fa" containerID="2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0" exitCode=0
Jul 30 01:47:47.113665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.113520    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"
Jul 30 01:47:47.113665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.113556    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerDied","Data":"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"}
Jul 30 01:47:47.113665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.113613    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g" event={"ID":"8de1a192-5a5f-4424-958a-f05f991821fa","Type":"ContainerDied","Data":"a54671fdfb2d4a006edff327ae67ae7f8fce38ec7284d1b6a913d506a5c558c0"}
Jul 30 01:47:47.113665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.113637    2584 scope.go:117] "RemoveContainer" containerID="2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"
Jul 30 01:47:47.123821 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.123782    2584 scope.go:117] "RemoveContainer" containerID="1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"
Jul 30 01:47:47.132114 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.132095    2584 scope.go:117] "RemoveContainer" containerID="2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"
Jul 30 01:47:47.132369 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:47:47.132345    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0\": container with ID starting with 2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0 not found: ID does not exist" containerID="2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"
Jul 30 01:47:47.132427 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.132377    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0"} err="failed to get container status \"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0\": rpc error: code = NotFound desc = could not find container \"2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0\": container with ID starting with 2feb2a56303df89e40f58ba94f498126e42176453acade93fe155f19e43790e0 not found: ID does not exist"
Jul 30 01:47:47.132427 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.132394    2584 scope.go:117] "RemoveContainer" containerID="1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"
Jul 30 01:47:47.132644 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:47:47.132626    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a\": container with ID starting with 1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a not found: ID does not exist" containerID="1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"
Jul 30 01:47:47.132681 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.132651    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a"} err="failed to get container status \"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a\": rpc error: code = NotFound desc = could not find container \"1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a\": container with ID starting with 1710f098070005a3cf8b5faa287c6923a47a5ce3f3a04f28d039c3b6c5aa2e0a not found: ID does not exist"
Jul 30 01:47:47.145506 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.145485    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:47:47.167132 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.167108    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-b64c39-openshift-default-fdd4855d6-5st2g"]
Jul 30 01:47:47.385861 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:47.385829    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" path="/var/lib/kubelet/pods/8de1a192-5a5f-4424-958a-f05f991821fa/volumes"
Jul 30 01:47:55.641353 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:55.641274    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:47:59.231445 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.231402    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:47:59.231866 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.231664    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy" containerID="cri-o://5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6" gracePeriod=30
Jul 30 01:47:59.571091 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571051    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"]
Jul 30 01:47:59.571392 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571381    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c168ffa-3f8f-43d7-b166-e032b3c85086" containerName="main"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571394    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c168ffa-3f8f-43d7-b166-e032b3c85086" containerName="main"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571409    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5584889-b28e-4e6e-b16c-c2923c6a3e3a" containerName="revoke-keys"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571414    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5584889-b28e-4e6e-b16c-c2923c6a3e3a" containerName="revoke-keys"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571426    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571432    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571439    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571443 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571444    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571454    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571459    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571510    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571517    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5584889-b28e-4e6e-b16c-c2923c6a3e3a" containerName="revoke-keys"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571524    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571531    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de1a192-5a5f-4424-958a-f05f991821fa" containerName="istio-proxy"
Jul 30 01:47:59.571656 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.571537    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c168ffa-3f8f-43d7-b166-e032b3c85086" containerName="main"
Jul 30 01:47:59.574263 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.574246    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:47:59.576522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.576495    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 01:47:59.576522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.576513    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:47:59.580976 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.580951    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"]
Jul 30 01:47:59.675182 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.675150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w\" (UID: \"1041bdb8-4f4e-4ac9-83a1-100128cf0400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:47:59.776552 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.776512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w\" (UID: \"1041bdb8-4f4e-4ac9-83a1-100128cf0400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:47:59.777233 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.777212    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w\" (UID: \"1041bdb8-4f4e-4ac9-83a1-100128cf0400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:47:59.886860 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:47:59.886762    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:48:00.012847 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:00.012774    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"]
Jul 30 01:48:00.015320 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:48:00.015289    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1041bdb8_4f4e_4ac9_83a1_100128cf0400.slice/crio-b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5 WatchSource:0}: Error finding container b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5: Status 404 returned error can't find the container with id b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5
Jul 30 01:48:00.176412 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:00.176319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w" event={"ID":"1041bdb8-4f4e-4ac9-83a1-100128cf0400","Type":"ContainerStarted","Data":"b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5"}
Jul 30 01:48:01.182303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:01.182272    2584 generic.go:356] "Generic (PLEG): container finished" podID="1041bdb8-4f4e-4ac9-83a1-100128cf0400" containerID="8eb4cc65e4648e366b18013163d77abf5b5d3d19c1ea416338c46c7d69f9ee10" exitCode=0
Jul 30 01:48:01.182691 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:01.182357    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w" event={"ID":"1041bdb8-4f4e-4ac9-83a1-100128cf0400","Type":"ContainerDied","Data":"8eb4cc65e4648e366b18013163d77abf5b5d3d19c1ea416338c46c7d69f9ee10"}
Jul 30 01:48:02.314378 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:02.314355    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:48:02.402301 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:02.402269    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca\") pod \"1041bdb8-4f4e-4ac9-83a1-100128cf0400\" (UID: \"1041bdb8-4f4e-4ac9-83a1-100128cf0400\") "
Jul 30 01:48:02.402671 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:02.402647    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1041bdb8-4f4e-4ac9-83a1-100128cf0400" (UID: "1041bdb8-4f4e-4ac9-83a1-100128cf0400"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:48:02.503942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:02.503901    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1041bdb8-4f4e-4ac9-83a1-100128cf0400-maas-api-service-ca\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:03.196352 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:03.196313    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"
Jul 30 01:48:03.196521 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:03.196302    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w" event={"ID":"1041bdb8-4f4e-4ac9-83a1-100128cf0400","Type":"ContainerDied","Data":"b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5"}
Jul 30 01:48:03.196521 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:03.196465    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3f48ae4c8be0a76c08f57c4947b780b0d42fb9b68469138d625960ebece76e5"
Jul 30 01:48:04.488856 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.488819    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:48:04.622745 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622644    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.622745 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622683    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.622745 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622723    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622756    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622783    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcsk6\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622848    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622889    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622917    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623045 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.622941    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs\") pod \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\" (UID: \"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5\") "
Jul 30 01:48:04.623350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623021    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:48:04.623350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623082    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:48:04.623350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623308    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:48:04.623350 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623338    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-credential-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.623560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623359    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-socket\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.623560 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.623443    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:48:04.624064 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.624035    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:48:04.625159 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.625138    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 01:48:04.625254 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.625195    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6" (OuterVolumeSpecName: "kube-api-access-gcsk6") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "kube-api-access-gcsk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:48:04.625400 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.625383    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:48:04.625467 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.625404    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" (UID: "e5c8c798-1383-4165-92ad-2a8a2cd7d5b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:48:04.723946 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723913    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-envoy\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.723946 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723940    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-podinfo\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.723946 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723951    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcsk6\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-kube-api-access-gcsk6\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.724168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723960    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istiod-ca-cert\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.724168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723969    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-data\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.724168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723976    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-istio-token\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:04.724168 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:04.723983    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5-workload-certs\") on node \"ip-10-0-134-28.ec2.internal\" DevicePath \"\""
Jul 30 01:48:05.206736 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.206703    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerID="5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6" exitCode=0
Jul 30 01:48:05.206928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.206752    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"
Jul 30 01:48:05.206928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.206769    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerDied","Data":"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"}
Jul 30 01:48:05.206928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.206836    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn" event={"ID":"e5c8c798-1383-4165-92ad-2a8a2cd7d5b5","Type":"ContainerDied","Data":"8d0fb318dfa520aa73d23199c6f74a0326b1b6ee2829e7bc4a6855bbd21b2334"}
Jul 30 01:48:05.206928 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.206860    2584 scope.go:117] "RemoveContainer" containerID="5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"
Jul 30 01:48:05.215773 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.215753    2584 scope.go:117] "RemoveContainer" containerID="70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"
Jul 30 01:48:05.224008 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.223988    2584 scope.go:117] "RemoveContainer" containerID="5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"
Jul 30 01:48:05.224298 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:48:05.224278    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6\": container with ID starting with 5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6 not found: ID does not exist" containerID="5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"
Jul 30 01:48:05.224381 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.224307    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6"} err="failed to get container status \"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6\": rpc error: code = NotFound desc = could not find container \"5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6\": container with ID starting with 5a8f193c191c5949881b9f59fe41d556c977f89b625fab64826395eeb9310dd6 not found: ID does not exist"
Jul 30 01:48:05.224381 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.224325    2584 scope.go:117] "RemoveContainer" containerID="70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"
Jul 30 01:48:05.224592 ip-10-0-134-28 kubenswrapper[2584]: E0730 01:48:05.224575    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52\": container with ID starting with 70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52 not found: ID does not exist" containerID="70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"
Jul 30 01:48:05.224637 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.224597    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52"} err="failed to get container status \"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52\": rpc error: code = NotFound desc = could not find container \"70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52\": container with ID starting with 70376eb9cb7e87f00a61c146671548bd31a7dbd56edab0467eccdfab9a750d52 not found: ID does not exist"
Jul 30 01:48:05.229209 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.229184    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:48:05.233470 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.233449    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-55b321-openshift-default-6448f9b46d-g57vn"]
Jul 30 01:48:05.385040 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:05.385007    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" path="/var/lib/kubelet/pods/e5c8c798-1383-4165-92ad-2a8a2cd7d5b5/volumes"
Jul 30 01:48:10.141085 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:10.141052    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"]
Jul 30 01:48:10.144333 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:10.144304    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fba93c-bnm8w"]
Jul 30 01:48:11.385502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:11.385457    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1041bdb8-4f4e-4ac9-83a1-100128cf0400" path="/var/lib/kubelet/pods/1041bdb8-4f4e-4ac9-83a1-100128cf0400/volumes"
Jul 30 01:48:12.641078 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:12.641039    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-686rb"]
Jul 30 01:48:17.181942 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:17.181907    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7545554d8c-7l8hw_d2de6e24-4bb1-42e2-b164-97bb38507006/authorino/0.log"
Jul 30 01:48:23.260265 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:23.260230    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-hlfzf_5807f19c-c3c2-443e-a7e1-fdbc88553aa0/manager/0.log"
Jul 30 01:48:23.265533 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:23.265511    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-hlfzf_5807f19c-c3c2-443e-a7e1-fdbc88553aa0/copy-manifests/0.log"
Jul 30 01:48:23.830109 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:23.830083    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756235-2qb9r_44afff0b-34b1-4283-ace6-0b6e3049d5f0/cleanup/0.log"
Jul 30 01:48:23.937603 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:23.937574    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756250-dwdns_f61533aa-e622-4dfa-bb6f-418a277c7c0f/cleanup/0.log"
Jul 30 01:48:25.420107 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:25.420078    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7545554d8c-7l8hw_d2de6e24-4bb1-42e2-b164-97bb38507006/authorino/0.log"
Jul 30 01:48:25.759144 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:25.759113    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jsfc4_2a96e726-75b5-41de-82be-751a40e9e09f/kuadrant-console-plugin/0.log"
Jul 30 01:48:26.114522 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:26.114444    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-686rb_69b340f6-26de-4eca-920b-d57af1bd7b52/limitador/0.log"
Jul 30 01:48:26.803860 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:26.803826    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-kmw2m_f05029d5-2f2f-4cf3-a132-a600cdb23b7d/kube-auth-proxy/0.log"
Jul 30 01:48:28.263737 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:28.263709    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bbtgd_3fb369e4-79ed-476c-956c-3fca5a4ce041/main/0.log"
Jul 30 01:48:36.360592 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:36.360559    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j69sg_b10cef9e-bbe3-45a5-aefd-30c11fb84b4d/global-pull-secret-syncer/0.log"
Jul 30 01:48:36.446564 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:36.446535    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pjvpj_13569e51-ed67-4de6-b024-bee5667e5212/konnectivity-agent/0.log"
Jul 30 01:48:36.523551 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:36.523522    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-28.ec2.internal_f0f6325ce07165ebf2e9979367b54cf4/haproxy/0.log"
Jul 30 01:48:41.868618 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:41.868580    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7545554d8c-7l8hw_d2de6e24-4bb1-42e2-b164-97bb38507006/authorino/0.log"
Jul 30 01:48:41.947989 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:41.947915    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jsfc4_2a96e726-75b5-41de-82be-751a40e9e09f/kuadrant-console-plugin/0.log"
Jul 30 01:48:42.171570 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:42.171541    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-686rb_69b340f6-26de-4eca-920b-d57af1bd7b52/limitador/0.log"
Jul 30 01:48:44.100748 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.100714    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-hvnbr_156c239b-160e-4477-a71c-4ad6bca201af/cluster-monitoring-operator/0.log"
Jul 30 01:48:44.256469 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.256442    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fqcv_8c9e072d-42ce-4fc5-8a5a-2b91f2842f06/node-exporter/0.log"
Jul 30 01:48:44.278076 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.278051    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fqcv_8c9e072d-42ce-4fc5-8a5a-2b91f2842f06/kube-rbac-proxy/0.log"
Jul 30 01:48:44.298230 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.298204    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fqcv_8c9e072d-42ce-4fc5-8a5a-2b91f2842f06/init-textfile/0.log"
Jul 30 01:48:44.525506 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.525482    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rtr87_60524341-05a3-4fee-8847-6e354d55b589/kube-rbac-proxy-main/0.log"
Jul 30 01:48:44.547484 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.547459    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rtr87_60524341-05a3-4fee-8847-6e354d55b589/kube-rbac-proxy-self/0.log"
Jul 30 01:48:44.568077 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.568049    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rtr87_60524341-05a3-4fee-8847-6e354d55b589/openshift-state-metrics/0.log"
Jul 30 01:48:44.601301 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.601275    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/prometheus/0.log"
Jul 30 01:48:44.623058 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.623032    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/config-reloader/0.log"
Jul 30 01:48:44.649747 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.649724    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/thanos-sidecar/0.log"
Jul 30 01:48:44.669983 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.669963    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/kube-rbac-proxy-web/0.log"
Jul 30 01:48:44.691463 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.691439    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/kube-rbac-proxy/0.log"
Jul 30 01:48:44.713085 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.713060    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/kube-rbac-proxy-thanos/0.log"
Jul 30 01:48:44.740433 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.740411    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_70ea5d4f-dbaf-41c1-86cd-f597e66042f0/init-config-reloader/0.log"
Jul 30 01:48:44.768405 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.768382    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h4z85_f08ef237-f02a-42b4-b935-6abb62d0eaf9/prometheus-operator/0.log"
Jul 30 01:48:44.794420 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.794355    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h4z85_f08ef237-f02a-42b4-b935-6abb62d0eaf9/kube-rbac-proxy/0.log"
Jul 30 01:48:44.818502 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:44.818476    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-l4k7m_fa0a4fb4-4ed1-4dd9-85d2-7f0e12416856/prometheus-operator-admission-webhook/0.log"
Jul 30 01:48:46.416437 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416404    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"]
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416712    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416722    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416732    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416738    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416755    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1041bdb8-4f4e-4ac9-83a1-100128cf0400" containerName="revoke-keys"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416761    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1041bdb8-4f4e-4ac9-83a1-100128cf0400" containerName="revoke-keys"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416769    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416774    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416861    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c8c798-1383-4165-92ad-2a8a2cd7d5b5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416871    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be293cf-7440-4744-9a4d-16f568b49db5" containerName="istio-proxy"
Jul 30 01:48:46.416895 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.416877    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1041bdb8-4f4e-4ac9-83a1-100128cf0400" containerName="revoke-keys"
Jul 30 01:48:46.421099 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.421078    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.423319 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.423296    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwqhr\"/\"default-dockercfg-hsst4\""
Jul 30 01:48:46.423442 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.423322    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"openshift-service-ca.crt\""
Jul 30 01:48:46.424361 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.424336    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"kube-root-ca.crt\""
Jul 30 01:48:46.429487 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.429464    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"]
Jul 30 01:48:46.497765 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.497725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j79fx\" (UniqueName: \"kubernetes.io/projected/1df03270-dd52-4bc9-92ca-82bd3ee05af4-kube-api-access-j79fx\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.497765 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.497763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-lib-modules\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.498036 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.497879    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-proc\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.498036 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.497929    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-sys\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.498144 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.498111    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-podres\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.598996 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.598952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-proc\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.598996 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-sys\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599047    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-podres\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599095    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j79fx\" (UniqueName: \"kubernetes.io/projected/1df03270-dd52-4bc9-92ca-82bd3ee05af4-kube-api-access-j79fx\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599103    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-proc\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599156    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-sys\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599171    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-lib-modules\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599252 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599214    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-podres\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.599450 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.599262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df03270-dd52-4bc9-92ca-82bd3ee05af4-lib-modules\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.607000 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.606977    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j79fx\" (UniqueName: \"kubernetes.io/projected/1df03270-dd52-4bc9-92ca-82bd3ee05af4-kube-api-access-j79fx\") pod \"perf-node-gather-daemonset-5fkd6\" (UID: \"1df03270-dd52-4bc9-92ca-82bd3ee05af4\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.731711 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.731599    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:46.862646 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:46.862576    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"]
Jul 30 01:48:46.864944 ip-10-0-134-28 kubenswrapper[2584]: W0730 01:48:46.864912    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1df03270_dd52_4bc9_92ca_82bd3ee05af4.slice/crio-d58d691b3e3fb2724d580e61b53393fdc150e355de0d60f52e36f541b8a7c348 WatchSource:0}: Error finding container d58d691b3e3fb2724d580e61b53393fdc150e355de0d60f52e36f541b8a7c348: Status 404 returned error can't find the container with id d58d691b3e3fb2724d580e61b53393fdc150e355de0d60f52e36f541b8a7c348
Jul 30 01:48:47.403751 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:47.403714    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6" event={"ID":"1df03270-dd52-4bc9-92ca-82bd3ee05af4","Type":"ContainerStarted","Data":"d03f91b4f62bb38ba8b8f16e05d5a20a8b5e249685621919d74c87c3db9c86b3"}
Jul 30 01:48:47.403981 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:47.403771    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6" event={"ID":"1df03270-dd52-4bc9-92ca-82bd3ee05af4","Type":"ContainerStarted","Data":"d58d691b3e3fb2724d580e61b53393fdc150e355de0d60f52e36f541b8a7c348"}
Jul 30 01:48:47.403981 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:47.403829    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:47.418680 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:47.418625    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6" podStartSLOduration=1.418611013 podStartE2EDuration="1.418611013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 01:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:48:47.417549963 +0000 UTC m=+3914.608498379" watchObservedRunningTime="2026-07-30 01:48:47.418611013 +0000 UTC m=+3914.609559344"
Jul 30 01:48:47.936924 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:47.936892    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-s6m82_9971f216-379b-4067-95ca-34ce25bbea27/volume-data-source-validator/0.log"
Jul 30 01:48:49.029539 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:49.029509    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t69pf_dc4b2be7-6a3d-4f54-acfc-37a1d42b5754/dns/0.log"
Jul 30 01:48:49.056586 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:49.056556    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t69pf_dc4b2be7-6a3d-4f54-acfc-37a1d42b5754/kube-rbac-proxy/0.log"
Jul 30 01:48:49.081704 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:49.081678    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-62tnh_86d15d74-beff-4568-9c89-660ad1fca368/dns-node-resolver/0.log"
Jul 30 01:48:49.751321 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:49.751290    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7fc6ddfd74-lbz2m_91cb2a4f-8c53-4ae4-be1d-07eac2de3dcd/registry/0.log"
Jul 30 01:48:49.845268 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:49.845236    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xv4mc_1301c9a9-7463-4730-9548-8fc0a02803cb/node-ca/0.log"
Jul 30 01:48:50.957959 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:50.957933    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-kmw2m_f05029d5-2f2f-4cf3-a132-a600cdb23b7d/kube-auth-proxy/0.log"
Jul 30 01:48:51.952089 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:51.952058    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxzv5_0f37938d-32a7-444e-b325-ea4e94f96f31/serve-healthcheck-canary/0.log"
Jul 30 01:48:52.574140 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:52.574111    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tjr6x_fe11ab37-4387-4566-ae2c-3febc2522285/kube-rbac-proxy/0.log"
Jul 30 01:48:52.593994 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:52.593970    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tjr6x_fe11ab37-4387-4566-ae2c-3febc2522285/exporter/0.log"
Jul 30 01:48:52.614303 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:52.614279    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tjr6x_fe11ab37-4387-4566-ae2c-3febc2522285/extractor/0.log"
Jul 30 01:48:53.418070 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:53.418044    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-5fkd6"
Jul 30 01:48:55.708527 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:55.708501    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-hlfzf_5807f19c-c3c2-443e-a7e1-fdbc88553aa0/manager/0.log"
Jul 30 01:48:55.733229 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:55.733209    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-hlfzf_5807f19c-c3c2-443e-a7e1-fdbc88553aa0/copy-manifests/0.log"
Jul 30 01:48:57.163573 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:57.163541    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d674c9db-vvjct_8b3042e7-0fa9-49cb-955c-6ed694c5a08c/manager/0.log"
Jul 30 01:48:57.254059 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:57.254029    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-kmw2m_f05029d5-2f2f-4cf3-a132-a600cdb23b7d/kube-auth-proxy/0.log"
Jul 30 01:48:58.977474 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:48:58.977441    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-kmw2m_f05029d5-2f2f-4cf3-a132-a600cdb23b7d/kube-auth-proxy/0.log"
Jul 30 01:49:02.525851 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:02.525814    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp_cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de/kube-storage-version-migrator-operator/1.log"
Jul 30 01:49:02.526786 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:02.526764    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-54hcp_cfbf34d4-88f0-4f0c-ba96-b5f5ef8147de/kube-storage-version-migrator-operator/0.log"
Jul 30 01:49:03.771760 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.771732    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/kube-multus-additional-cni-plugins/0.log"
Jul 30 01:49:03.792232 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.792213    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/egress-router-binary-copy/0.log"
Jul 30 01:49:03.813579 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.813558    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/cni-plugins/0.log"
Jul 30 01:49:03.832665 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.832645    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/bond-cni-plugin/0.log"
Jul 30 01:49:03.852472 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.852454    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/routeoverride-cni/0.log"
Jul 30 01:49:03.875260 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.875238    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/whereabouts-cni-bincopy/0.log"
Jul 30 01:49:03.895368 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:03.895338    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xnfrk_e3013612-b6cd-4e9f-9497-b4a528e08842/whereabouts-cni/0.log"
Jul 30 01:49:04.024517 ip-10-0-134-28 kubenswrapper[2584]: I0730 01:49:04.024436    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zd5pd_45751b95-3ef0-4dd9-9a0c-75ee1f348f2f/kube-multus/0.log"