Jul 28 10:21:06.738985 ip-10-0-129-31 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 10:21:06.738991 ip-10-0-129-31 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 10:21:06.738998 ip-10-0-129-31 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 10:21:06.739216 ip-10-0-129-31 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 10:21:18.102257 ip-10-0-129-31 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 10:21:18.102275 ip-10-0-129-31 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 072dec48316a4eb5bf362544d91cec49 --
Jul 28 10:22:54.596897 ip-10-0-129-31 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 10:22:55.026276 ip-10-0-129-31 kubenswrapper[2544]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:22:55.026276 ip-10-0-129-31 kubenswrapper[2544]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 10:22:55.026276 ip-10-0-129-31 kubenswrapper[2544]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:22:55.026276 ip-10-0-129-31 kubenswrapper[2544]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 10:22:55.026276 ip-10-0-129-31 kubenswrapper[2544]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:22:55.028467 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.028316    2544 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 10:22:55.035978 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035958    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.035978 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035975    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.035978 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035979    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.035978 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035982    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.035978 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035985    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035988    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035991    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035994    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035997    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.035999    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036002    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036004    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036007    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036010    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036012    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036015    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036019    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036022    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036025    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036031    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036034    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036037    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036040    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036043    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.036172 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036045    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036048    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036050    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036053    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036055    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036066    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036069    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036071    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036074    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036079    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036083    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036086    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036089    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036092    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036095    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036098    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036101    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036104    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036107    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.036694 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036110    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036113    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036115    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036118    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036121    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036123    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036126    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036130    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036133    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036135    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036138    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036140    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036143    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036145    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036148    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036150    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036153    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036155    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036158    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036162    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036165    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.037151 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036168    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036170    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036172    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036175    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036178    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036181    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036184    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036188    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036191    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036193    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036197    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036199    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036202    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036204    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036208    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036211    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036214    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036216    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036219    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036222    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.037764 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036225    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036227    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036230    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036232    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036235    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036237    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036240    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036242    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036245    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036247    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036250    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036252    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036256    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036260    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036264    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036980    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036988    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036991    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036996    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.038252 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.036998    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037002    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037004    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037007    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037011    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037015    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037018    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037023    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037026    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037029    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037031    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037035    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037039    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037043    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037046    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037049    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037052    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037054    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.038717 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037057    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037059    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037062    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037064    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037067    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037069    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037072    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037074    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037077    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037079    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037082    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037084    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037087    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037090    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037093    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037095    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037098    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037100    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037103    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037105    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.039160 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037107    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037110    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037113    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037116    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037118    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037121    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037123    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037125    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037128    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037130    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037133    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037135    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037138    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037140    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037142    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037145    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037147    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037149    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037152    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037155    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.039664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037157    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037160    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037162    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037165    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037168    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037171    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037174    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037176    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037179    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037181    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037185    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037187    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037190    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037192    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037195    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037197    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037200    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037202    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037204    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037207    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.040155 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037209    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037213    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037215    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037217    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037220    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037223    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037225    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037227    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037230    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037232    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037235    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037237    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037240    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037242    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037245    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037247    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037250    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037613    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037618    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.040662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037621    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037623    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037626    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037628    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037631    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037633    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037636    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037638    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037641    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037644    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037646    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037649    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037651    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037654    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037657    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037662    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037665    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037668    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037671    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037674    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.041145 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037677    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037680    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037682    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037685    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037687    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037690    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037693    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037695    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037698    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037700    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037703    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037706    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037709    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037711    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037714    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037716    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037718    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037721    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037723    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037726    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.041629 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037728    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037731    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037734    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037736    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037739    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037741    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037743    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037748    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037751    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037754    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037757    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037760    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037762    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037765    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037767    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037770    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037772    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037775    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037781    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.042124 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037783    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037786    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037790    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037793    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037796    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037798    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037801    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037804    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037806    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037809    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037811    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037813    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037816    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037818    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037821    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037823    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037826    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037828    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037832    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.042638 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037835    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037837    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037842    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037844    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037846    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037849    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037851    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037854    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037856    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037859    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037862    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037864    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037871    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037873    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037876    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037878    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037881    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037883    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.037885    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037947    2544 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 10:22:55.043096 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037953    2544 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037958    2544 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037962    2544 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037966    2544 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037968    2544 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037972    2544 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037976    2544 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037979    2544 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037982    2544 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037985    2544 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037988    2544 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037991    2544 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037994    2544 flags.go:64] FLAG: --cgroup-root=""
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.037997    2544 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038000    2544 flags.go:64] FLAG: --client-ca-file=""
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038003    2544 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038006    2544 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038010    2544 flags.go:64] FLAG: --cluster-domain=""
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038012    2544 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038015    2544 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038018    2544 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038022    2544 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038025    2544 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 10:22:55.043598 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038028    2544 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038033    2544 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038036    2544 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038039    2544 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038042    2544 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038045    2544 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038048    2544 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038051    2544 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038055    2544 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038057    2544 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038060    2544 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038063    2544 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038066    2544 flags.go:64] FLAG: --enable-server="true"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038068    2544 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038072    2544 flags.go:64] FLAG: --event-burst="100"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038075    2544 flags.go:64] FLAG: --event-qps="50"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038078    2544 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038080    2544 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038083    2544 flags.go:64] FLAG: --eviction-hard=""
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038091    2544 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038094    2544 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038097    2544 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038100    2544 flags.go:64] FLAG: --eviction-soft=""
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038103    2544 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038105    2544 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 10:22:55.044183 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038109    2544 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038111    2544 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038114    2544 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038117    2544 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038120    2544 flags.go:64] FLAG: --feature-gates=""
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038123    2544 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038126    2544 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038129    2544 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038134    2544 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038137    2544 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038140    2544 flags.go:64] FLAG: --help="false"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038143    2544 flags.go:64] FLAG: --hostname-override="ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038146    2544 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038149    2544 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038152    2544 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038155    2544 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038158    2544 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038161    2544 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038164    2544 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038166    2544 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038169    2544 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038172    2544 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038175    2544 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038178    2544 flags.go:64] FLAG: --kube-reserved=""
Jul 28 10:22:55.044824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038180    2544 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038183    2544 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038186    2544 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038188    2544 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038191    2544 flags.go:64] FLAG: --lock-file=""
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038194    2544 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038196    2544 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038200    2544 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038205    2544 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038208    2544 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038210    2544 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038213    2544 flags.go:64] FLAG: --logging-format="text"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038216    2544 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038219    2544 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038221    2544 flags.go:64] FLAG: --manifest-url=""
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038224    2544 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038230    2544 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038233    2544 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038237    2544 flags.go:64] FLAG: --max-pods="110"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038240    2544 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038243    2544 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038245    2544 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038248    2544 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038251    2544 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038254    2544 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 10:22:55.045399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038257    2544 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038264    2544 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038267    2544 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038270    2544 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038273    2544 flags.go:64] FLAG: --pod-cidr=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038276    2544 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038281    2544 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038284    2544 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038287    2544 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038290    2544 flags.go:64] FLAG: --port="10250"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038293    2544 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038295    2544 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fab91279379c0484"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038298    2544 flags.go:64] FLAG: --qos-reserved=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038301    2544 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038304    2544 flags.go:64] FLAG: --register-node="true"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038307    2544 flags.go:64] FLAG: --register-with-taints=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038310    2544 flags.go:64] FLAG: --registry-burst="10"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038313    2544 flags.go:64] FLAG: --registry-qps="5"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038316    2544 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038319    2544 flags.go:64] FLAG: --reserved-memory=""
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038323    2544 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038326    2544 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038328    2544 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038332    2544 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038336    2544 flags.go:64] FLAG: --runonce="false"
Jul 28 10:22:55.046037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038339    2544 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038342    2544 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038344    2544 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038347    2544 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038350    2544 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038353    2544 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038356    2544 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038359    2544 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038362    2544 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038364    2544 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038367    2544 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038370    2544 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038372    2544 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038375    2544 flags.go:64] FLAG: --system-cgroups=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038378    2544 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038383    2544 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038385    2544 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038388    2544 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038392    2544 flags.go:64] FLAG: --tls-min-version=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038395    2544 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038398    2544 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038400    2544 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038403    2544 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038406    2544 flags.go:64] FLAG: --v="2"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038409    2544 flags.go:64] FLAG: --version="false"
Jul 28 10:22:55.046646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038416    2544 flags.go:64] FLAG: --vmodule=""
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038420    2544 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.038423    2544 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038528    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038531    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038534    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038538    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038541    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038543    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038546    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038548    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038551    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038553    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038570    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038573    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038575    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038578    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038580    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038583    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038586    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.047258 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038589    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038591    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038594    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038597    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038599    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038602    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038604    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038607    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038610    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038612    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038615    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038618    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038620    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038623    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038626    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038629    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038631    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038635    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038641    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038644    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.047790 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038647    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038649    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038653    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038656    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038659    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038662    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038665    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038667    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038670    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038672    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038675    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038678    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038680    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038683    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038686    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038688    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038691    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038693    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038697    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038700    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.048294 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038703    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038706    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038709    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038711    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038714    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038717    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038720    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038723    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038725    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038728    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038732    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038735    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038737    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038740    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038742    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038745    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038747    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038750    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038752    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.048792 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038755    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038757    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038760    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038762    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038765    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038767    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038769    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038772    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038774    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038777    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038779    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038782    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038784    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038786    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038789    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038791    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038794    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038796    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038798    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.049261 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038801    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038803    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038806    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.038810    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.039623    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.046431    2544 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.046444    2544 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.046462    2544 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.046471    2544 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046536    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046541    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046544    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046548    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046550    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046553    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.049794 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046571    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046573    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046576    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046578    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046581    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046584    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046586    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046589    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046592    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046594    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046597    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046600    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046603    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046605    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046607    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046610    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046614    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046618    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046621    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046623    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.050168 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046626    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046629    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046632    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046636    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046638    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046641    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046644    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046646    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046649    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046651    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046654    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046657    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046659    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046662    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046665    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046667    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046670    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046672    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046675    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046677    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.050687 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046679    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046682    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046685    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046687    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046689    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046692    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046694    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046697    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046699    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046701    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046704    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046706    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046709    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046712    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046715    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046718    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046721    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046724    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046726    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046729    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.051216 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046731    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046734    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046738    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046741    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046744    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046747    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046749    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046752    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046754    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046756    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046759    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046762    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046764    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046767    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046769    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046772    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046774    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046776    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046779    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.051726 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046781    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046784    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046786    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046788    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046791    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046793    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046796    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046798    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046801    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046805    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046809    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046812    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046815    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046817    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.052190 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.046822    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046951    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046956    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046959    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046962    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046965    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046967    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046970    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046972    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046975    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046978    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046980    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046983    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046985    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046988    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046991    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046993    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046995    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.046998    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047000    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:22:55.052531 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047002    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047005    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047007    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047009    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047012    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047014    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047017    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047019    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047022    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047025    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047027    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047030    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047032    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047035    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047037    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047040    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047043    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047045    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047047    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047050    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:22:55.053031 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047052    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047054    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047057    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047059    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047062    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047066    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047069    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047071    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047074    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047077    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047079    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047081    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047084    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047086    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047089    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047091    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047093    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047096    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047099    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047101    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:22:55.053513 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047104    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047106    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047109    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047112    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047114    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047116    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047118    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047121    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047124    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047127    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047130    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047132    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047135    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047137    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047139    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047142    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047144    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047147    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047149    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:22:55.054026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047152    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047154    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047157    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047161    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047164    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047167    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047169    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047172    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047175    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047178    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047180    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047183    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047186    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047189    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047191    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047194    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047202    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047205    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047208    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:22:55.054530 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047210    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:55.047212    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.047217    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.047317    2544 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.049655    2544 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.051324    2544 server.go:1019] "Starting client certificate rotation"
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.051438    2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.052287    2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 10:22:55.055003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.054392    2544 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 10:22:55.058490 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.058472    2544 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 10:22:55.072395 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.072378    2544 log.go:25] "Validated CRI v1 runtime API"
Jul 28 10:22:55.074824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.074811    2544 log.go:25] "Validated CRI v1 image API"
Jul 28 10:22:55.076536 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.076519    2544 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 10:22:55.080104 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.080083    2544 fs.go:135] Filesystem UUIDs: map[3de2d915-a66c-4893-b4d2-23c0a143ff52:/dev/nvme0n1p3 6a99755f-cad3-43cc-b62b-7aed674f2d13:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 28 10:22:55.080172 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.080102    2544 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 10:22:55.085065 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.085045    2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 10:22:55.086869 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.086768    2544 manager.go:216] Machine: {Timestamp:2026-07-28 10:22:55.085422016 +0000 UTC m=+0.363359180 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096595 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2910e2775885765df4176987aaae4f SystemUUID:ec2910e2-7758-8576-5df4-176987aaae4f BootID:072dec48-316a-4eb5-bf36-2544d91cec49 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ea:15:62:6e:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ea:15:62:6e:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:b7:46:5e:1a:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 10:22:55.086869 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.086867    2544 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 10:22:55.086966 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.086936    2544 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 10:22:55.090267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.090248    2544 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 10:22:55.090395 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.090270    2544 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-31.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 10:22:55.091154 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.091144    2544 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 10:22:55.091187 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.091156    2544 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 10:22:55.091187 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.091163    2544 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 10:22:55.091942 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.091932    2544 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 10:22:55.093825 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.093816    2544 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 10:22:55.097206 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097195    2544 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 10:22:55.097314 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097304    2544 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 10:22:55.097345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097333    2544 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 10:22:55.097371 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097350    2544 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 10:22:55.097371 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097364    2544 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 10:22:55.097432 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097373    2544 kubelet.go:398] "Adding apiserver pod source"
Jul 28 10:22:55.097432 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097382    2544 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 10:22:55.097768 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097754    2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 10:22:55.097815 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.097780    2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 10:22:55.100526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.100509    2544 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 10:22:55.102129 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102116    2544 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 10:22:55.102179 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102146    2544 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 10:22:55.102943 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102931    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 10:22:55.102980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102953    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 10:22:55.102980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102959    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 10:22:55.102980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102965    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 10:22:55.102980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102973    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 10:22:55.102980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102980    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102989    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.102994    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103005    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103010    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103018    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 10:22:55.103098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103025    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 10:22:55.103876 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103861    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 10:22:55.103876 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.103875    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 10:22:55.107062 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.107047    2544 server.go:1268] "Started kubelet"
Jul 28 10:22:55.107194 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.107122    2544 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 10:22:55.107279 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.107168    2544 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 10:22:55.111754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.111725    2544 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 10:22:55.112223 ip-10-0-129-31 systemd[1]: Started Kubernetes Kubelet.
Jul 28 10:22:55.114187 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.114171    2544 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 10:22:55.114915 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.114864    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-31.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 10:22:55.115103 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.115077    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 10:22:55.115221 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.115198    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 10:22:55.115697 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.115679    2544 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 10:22:55.119219 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.118204    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-31.ec2.internal.18c66d0610243b6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-31.ec2.internal,UID:ip-10-0-129-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-31.ec2.internal,},FirstTimestamp:2026-07-28 10:22:55.107021675 +0000 UTC m=+0.384958840,LastTimestamp:2026-07-28 10:22:55.107021675 +0000 UTC m=+0.384958840,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-31.ec2.internal,}"
Jul 28 10:22:55.121462 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.121444    2544 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 10:22:55.123548 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.123532    2544 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 10:22:55.123641 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.123536    2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 10:22:55.123641 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.123631    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.124316 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124296    2544 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 10:22:55.124316 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124314    2544 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124300    2544 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124377    2544 factory.go:55] Registering systemd factory
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124393    2544 factory.go:223] Registration of the systemd container factory successfully
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124399    2544 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124407    2544 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124610    2544 factory.go:153] Registering CRI-O factory
Jul 28 10:22:55.124646 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124621    2544 factory.go:223] Registration of the crio container factory successfully
Jul 28 10:22:55.124944 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124659    2544 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 10:22:55.124944 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124679    2544 factory.go:103] Registering Raw factory
Jul 28 10:22:55.124944 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.124693    2544 manager.go:1196] Started watching for new ooms in manager
Jul 28 10:22:55.125227 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.125212    2544 manager.go:349] Starting recovery of all containers
Jul 28 10:22:55.125957 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.125920    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 10:22:55.126043 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.126027    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 10:22:55.136125 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.136113    2544 manager.go:354] Recovery completed
Jul 28 10:22:55.139911 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.139897    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.141851 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.141836    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.141908 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.141866    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.141908 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.141880    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.142454 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.142441    2544 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 10:22:55.142454 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.142452    2544 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 10:22:55.142589 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.142458    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.142589 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.142468    2544 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 10:22:55.143179 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143164    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.143237 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143195    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.143237 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143209    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.143681 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143668    2544 policy_none.go:49] "None policy: Start"
Jul 28 10:22:55.143729 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143686    2544 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 10:22:55.143729 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.143697    2544 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 10:22:55.144592 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.144582    2544 policy_none.go:47] "Start"
Jul 28 10:22:55.153553 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.153447    2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-542ws"
Jul 28 10:22:55.154326 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.154262    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-31.ec2.internal.18c66d061237b014  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-31.ec2.internal,UID:ip-10-0-129-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-31.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-31.ec2.internal,},FirstTimestamp:2026-07-28 10:22:55.141851156 +0000 UTC m=+0.419788320,LastTimestamp:2026-07-28 10:22:55.141851156 +0000 UTC m=+0.419788320,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-31.ec2.internal,}"
Jul 28 10:22:55.163591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.163574    2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-542ws"
Jul 28 10:22:55.165360 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.165303    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-31.ec2.internal.18c66d06123801c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-31.ec2.internal,UID:ip-10-0-129-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-31.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-31.ec2.internal,},FirstTimestamp:2026-07-28 10:22:55.141872072 +0000 UTC m=+0.419809240,LastTimestamp:2026-07-28 10:22:55.141872072 +0000 UTC m=+0.419809240,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-31.ec2.internal,}"
Jul 28 10:22:55.180736 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.180721    2544 manager.go:337] "Starting Device Plugin manager"
Jul 28 10:22:55.180821 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.180758    2544 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 10:22:55.180821 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.180767    2544 server.go:87] "Starting device plugin registration server"
Jul 28 10:22:55.180983 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.180971    2544 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 10:22:55.181019 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.180987    2544 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 10:22:55.181117 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.181101    2544 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 10:22:55.181218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.181204    2544 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 10:22:55.181218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.181217    2544 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 10:22:55.181586 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.181544    2544 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 10:22:55.181640 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.181621    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.261792 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.261771    2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 10:22:55.263085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.263068    2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 10:22:55.263162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.263107    2544 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 10:22:55.263162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.263125    2544 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 10:22:55.263232 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.263154    2544 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 10:22:55.265829 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.265812    2544 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:22:55.281913 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.281877    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.282592 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.282576    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.282649 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.282604    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.282649 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.282614    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.282649 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.282633    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.292079 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.292063    2544 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.292153 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.292081    2544 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-31.ec2.internal\": node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.332769 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.332749    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.364915 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.364897    2544 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"]
Jul 28 10:22:55.364980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.364948    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.365694 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.365678    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.365763 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.365708    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.365763 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.365719    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.367228 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367214    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.367398 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367384    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.367439 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367431    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.367867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367852    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.367932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367883    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.367932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367896    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.367932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367899    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.367932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367920    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.367932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.367929    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.368883 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.368871    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.368921 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.368894    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:22:55.369452 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.369439    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:22:55.369512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.369464    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:22:55.369512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.369477    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:22:55.407060 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.407036    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-31.ec2.internal\" not found" node="ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.410298 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.410282    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-31.ec2.internal\" not found" node="ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.426553 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.426532    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85547fd26d62ca170d8549b389f25e88-config\") pod \"kube-apiserver-proxy-ip-10-0-129-31.ec2.internal\" (UID: \"85547fd26d62ca170d8549b389f25e88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.426652 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.426578    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.426652 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.426603    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.433637 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.433623    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.527686 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527660    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.527686 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527686    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.527871 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527703    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85547fd26d62ca170d8549b389f25e88-config\") pod \"kube-apiserver-proxy-ip-10-0-129-31.ec2.internal\" (UID: \"85547fd26d62ca170d8549b389f25e88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.527871 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527739    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85547fd26d62ca170d8549b389f25e88-config\") pod \"kube-apiserver-proxy-ip-10-0-129-31.ec2.internal\" (UID: \"85547fd26d62ca170d8549b389f25e88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.527871 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527769    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.527871 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.527773    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5d45a3473f9607ff7c933ffd8b119c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal\" (UID: \"bf5d45a3473f9607ff7c933ffd8b119c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.533739 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.533700    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.634054 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.634036    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.709237 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.709217    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.713265 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:55.713113    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:55.734372 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.734345    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.834917 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.834866    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:55.935438 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:55.935412    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:56.035905 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:56.035881    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-31.ec2.internal\" not found"
Jul 28 10:22:56.050044 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.050025    2544 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 10:22:56.050155 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.050136    2544 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 10:22:56.099406 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.099362    2544 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:22:56.131876 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.131855    2544 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:22:56.168926 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.168890    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 10:17:55 +0000 UTC" deadline="2028-01-27 18:21:50.305299916 +0000 UTC"
Jul 28 10:22:56.168926 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.168915    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13159h58m54.13638718s"
Jul 28 10:22:56.204262 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.204230    2544 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:22:56.204769 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:56.204740    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85547fd26d62ca170d8549b389f25e88.slice/crio-7f68aa8f98b1ccdad242efa9599e11aa4a6d1f880c7da4558bdee3ca45845997 WatchSource:0}: Error finding container 7f68aa8f98b1ccdad242efa9599e11aa4a6d1f880c7da4558bdee3ca45845997: Status 404 returned error can't find the container with id 7f68aa8f98b1ccdad242efa9599e11aa4a6d1f880c7da4558bdee3ca45845997
Jul 28 10:22:56.207537 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:56.207517    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5d45a3473f9607ff7c933ffd8b119c.slice/crio-da603139869132a7f65d33c9344d33d5db1c9026a7b6e5a6004c9ec75e033afe WatchSource:0}: Error finding container da603139869132a7f65d33c9344d33d5db1c9026a7b6e5a6004c9ec75e033afe: Status 404 returned error can't find the container with id da603139869132a7f65d33c9344d33d5db1c9026a7b6e5a6004c9ec75e033afe
Jul 28 10:22:56.209972 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.209956    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 10:22:56.224161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.224142    2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:56.244620 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.244598    2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 10:22:56.245820 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.245806    2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal"
Jul 28 10:22:56.261037 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.261017    2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 10:22:56.266086 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.266052    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal" event={"ID":"bf5d45a3473f9607ff7c933ffd8b119c","Type":"ContainerStarted","Data":"da603139869132a7f65d33c9344d33d5db1c9026a7b6e5a6004c9ec75e033afe"}
Jul 28 10:22:56.267014 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:56.266997    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal" event={"ID":"85547fd26d62ca170d8549b389f25e88","Type":"ContainerStarted","Data":"7f68aa8f98b1ccdad242efa9599e11aa4a6d1f880c7da4558bdee3ca45845997"}
Jul 28 10:22:57.098709 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.098451    2544 apiserver.go:52] "Watching apiserver"
Jul 28 10:22:57.106581 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.106542    2544 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 10:22:57.106946 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.106925    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-9h54r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz","openshift-multus/multus-additional-cni-plugins-cj4zz","openshift-multus/multus-tgzfb","openshift-network-diagnostics/network-check-target-mzjkp","openshift-network-operator/iptables-alerter-x4tg6","openshift-ovn-kubernetes/ovnkube-node-w2pqh","kube-system/global-pull-secret-syncer-67ksr","kube-system/konnectivity-agent-gh8sh","kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fhq8q","openshift-dns/node-resolver-bwn8m","openshift-image-registry/node-ca-p4g9v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal"]
Jul 28 10:22:57.109631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.109609    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.111878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.111858    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.112946 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.112927    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qnts6\""
Jul 28 10:22:57.113044 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.112985    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 10:22:57.113107 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.112926    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.113162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.113106    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 10:22:57.113323 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.113305    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 10:22:57.113621 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.113602    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.113860 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.113841    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 10:22:57.114218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.114198    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.114624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.114593    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 10:22:57.115876 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.115858    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.116096 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.116081    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lsrk6\""
Jul 28 10:22:57.116481 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.116460    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.116619 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.116602    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.117194 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.117176    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 10:22:57.122992 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.122964    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:57.123079 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.123053    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:22:57.123863 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.123842    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 10:22:57.124026 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124012    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 10:22:57.124119 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124103    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rwtdh\""
Jul 28 10:22:57.124168 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124125    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 10:22:57.124218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124203    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wmkp9\""
Jul 28 10:22:57.124250 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124012    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.124312 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.123845    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.124377 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.124363    2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 10:22:57.126912 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.126884    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.129705 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.129520    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 10:22:57.130049 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.130008    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.132854 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.132836    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d5vlj\""
Jul 28 10:22:57.133047 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.133026    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.133238 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.133222    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.133525 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.133509    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.133624 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.133601    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:22:57.134335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134314    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-systemd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134430 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134359    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-cnibin\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.134430 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134392    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-system-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.134430 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134425    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134457    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0297b33b-d269-40d6-8cd8-dfb44e340b55-iptables-alerter-script\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134494    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26pnf\" (UniqueName: \"kubernetes.io/projected/0297b33b-d269-40d6-8cd8-dfb44e340b55-kube-api-access-26pnf\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134517    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-env-overrides\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134574    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-kubelet\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134597    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-netns\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134657 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134637    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-node-log\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134685    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134713    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-os-release\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134738    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-netns\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134822    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-conf-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134855    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-var-lib-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134882    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-ovn\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134923    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e953407-ddc7-4913-950e-9e4048297038-serviceca\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.134949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134947    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlbtk\" (UniqueName: \"kubernetes.io/projected/5e953407-ddc7-4913-950e-9e4048297038-kube-api-access-jlbtk\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.134971    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135035    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135071    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-socket-dir-parent\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135093    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-hostroot\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135123    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135146    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-config\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135215    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovn-node-metrics-cert\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135237    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7c49\" (UniqueName: \"kubernetes.io/projected/4888ca2e-c893-417a-b98d-ac0cd5048fa9-kube-api-access-j7c49\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135259    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e953407-ddc7-4913-950e-9e4048297038-host\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135280    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hrhk\" (UniqueName: \"kubernetes.io/projected/baeba705-ba1e-436f-8c46-e08479f06175-kube-api-access-9hrhk\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.135319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135304    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-cnibin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135492    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-multus\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135522    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-bin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135546    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-kubelet\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135587    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tqg9\" (UniqueName: \"kubernetes.io/projected/68323c15-590d-448b-a746-58f284493bcb-kube-api-access-8tqg9\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135620    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0297b33b-d269-40d6-8cd8-dfb44e340b55-host-slash\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135642    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-script-lib\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135667    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-systemd-units\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135691    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-bin\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135713    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-netd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135736    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-os-release\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135761    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135797    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.135842 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135826    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-slash\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135861    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-etc-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135892    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-log-socket\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135918    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135938    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-cni-binary-copy\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.135986    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-k8s-cni-cncf-io\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.136017    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-multus-daemon-config\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.136040    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.136072    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-system-cni-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.136093    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-multus-certs\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.136345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.136115    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-etc-kubernetes\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.137119 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.137100    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.137706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.137685    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.137923 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.137903    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.140007 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.139474    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s47n8\""
Jul 28 10:22:57.140007 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.139638    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.140007 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.139668    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9j7gw\""
Jul 28 10:22:57.140922 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.140353    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 10:22:57.140922 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.140576    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.140922 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.140583    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.142005 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.141989    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.143172 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.143039    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 10:22:57.143172 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.143082    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:22:57.143172 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.143111    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hp846\""
Jul 28 10:22:57.144278 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.144260    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f5hhx\""
Jul 28 10:22:57.144356 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.144276    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 10:22:57.144985 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.144868    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.144985 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.144925    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:22:57.145204 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.145188    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 10:22:57.178022 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.177994    2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 10:22:57.225689 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.225660    2544 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 10:22:57.236595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236454    2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vtxlx"
Jul 28 10:22:57.236758 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236737    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.236851 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236793    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-env-overrides\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.236851 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236823    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-kubelet\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.236964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236849    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-node-log\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.236964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236876    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.236964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236904    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-run\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.236964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236930    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpzm4\" (UniqueName: \"kubernetes.io/projected/68b3ffc0-350b-432f-8ec6-d036e68e6148-kube-api-access-vpzm4\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.236964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236955    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.236989    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-var-lib-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237016    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlbtk\" (UniqueName: \"kubernetes.io/projected/5e953407-ddc7-4913-950e-9e4048297038-kube-api-access-jlbtk\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237042    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237070    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237095    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-registration-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237122    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-hosts-file\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237148    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-config\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237173    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovn-node-metrics-cert\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237198    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7c49\" (UniqueName: \"kubernetes.io/projected/4888ca2e-c893-417a-b98d-ac0cd5048fa9-kube-api-access-j7c49\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237226    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hrhk\" (UniqueName: \"kubernetes.io/projected/baeba705-ba1e-436f-8c46-e08479f06175-kube-api-access-9hrhk\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237269    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-cnibin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237293    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-multus\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237327    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-lib-modules\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237351    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-tmp\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237384    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-bin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237418    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237444    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fms6w\" (UniqueName: \"kubernetes.io/projected/f5dc3097-a4c6-4167-b237-9533efadbc76-kube-api-access-fms6w\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237471    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-multus-daemon-config\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237499    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-script-lib\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237524    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-bin\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237551    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-os-release\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237577    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237603    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brjhp\" (UniqueName: \"kubernetes.io/projected/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kube-api-access-brjhp\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237673    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-modprobe-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237699    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.237747 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237727    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-slash\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237755    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-etc-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237782    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237809    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-cni-binary-copy\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237835    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-etc-kubernetes\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237859    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-host\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237883    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/603776a9-6511-4a16-9f19-0b33fa8c6772-agent-certs\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237908    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-tmp-dir\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237936    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237968    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-system-cni-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.237995    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-device-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238021    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/603776a9-6511-4a16-9f19-0b33fa8c6772-konnectivity-ca\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238050    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6lwb\" (UniqueName: \"kubernetes.io/projected/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-kube-api-access-l6lwb\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238076    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238106    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-systemd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238132    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-system-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238157    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-kubernetes\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.238591 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238184    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-netns\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238210    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-os-release\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238237    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-netns\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238263    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-conf-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238293    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tqg9\" (UniqueName: \"kubernetes.io/projected/68323c15-590d-448b-a746-58f284493bcb-kube-api-access-8tqg9\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238321    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-kubelet-config\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238349    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0297b33b-d269-40d6-8cd8-dfb44e340b55-iptables-alerter-script\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238377    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-ovn\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238403    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e953407-ddc7-4913-950e-9e4048297038-serviceca\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238430    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-socket-dir-parent\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238463    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-hostroot\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238491    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysconfig\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238518    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-conf\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238544    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-var-lib-kubelet\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238574    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238600    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e953407-ddc7-4913-950e-9e4048297038-host\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238626    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0297b33b-d269-40d6-8cd8-dfb44e340b55-host-slash\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.239386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238679    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-kubelet\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238706    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-sys\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238734    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-k8s-cni-cncf-io\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238764    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238796    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-systemd-units\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238825    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-netd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238850    2544 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.238852    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239064    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-sys-fs\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239091    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239125    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-tuned\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239148    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-kubelet\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239161    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-log-socket\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239212    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-socket-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239240    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-dbus\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239271    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-multus-certs\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239297    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-systemd\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239318    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-cnibin\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.240267 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239339    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26pnf\" (UniqueName: \"kubernetes.io/projected/0297b33b-d269-40d6-8cd8-dfb44e340b55-kube-api-access-26pnf\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239214    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-var-lib-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239602    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239663    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-env-overrides\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239655    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-log-socket\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239721    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-node-log\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239753    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239756    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239791    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-multus-certs\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239810    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-system-cni-dir\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239830    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-cnibin\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239867    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-systemd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239894    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-system-cni-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.239966    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-netns\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.240046    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-os-release\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.240092    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-conf-dir\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.240191    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.240835    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-multus-daemon-config\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.241175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.240934    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-cnibin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241103    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-script-lib\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241201    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-bin\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241285    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeba705-ba1e-436f-8c46-e08479f06175-os-release\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241411    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovnkube-config\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241746    2544 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241826    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.241971    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-slash\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.242102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.242032    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-etc-openvswitch\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.242496 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.242306    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-etc-kubernetes\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.242888    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeba705-ba1e-436f-8c46-e08479f06175-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243092    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-run-netns\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243191    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243708    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-multus\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243766    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-cni-bin\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243817    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-systemd-units\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.243859    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-host-cni-netd\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.244960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.244324    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68323c15-590d-448b-a746-58f284493bcb-cni-binary-copy\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.245416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245030    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4888ca2e-c893-417a-b98d-ac0cd5048fa9-run-ovn\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.245416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245107    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-multus-socket-dir-parent\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.245416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245184    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e953407-ddc7-4913-950e-9e4048297038-host\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.245416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245192    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-hostroot\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.245416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245400    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0297b33b-d269-40d6-8cd8-dfb44e340b55-host-slash\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.245742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245438    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0297b33b-d269-40d6-8cd8-dfb44e340b55-iptables-alerter-script\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.245742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245468    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-var-lib-kubelet\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.245742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245507    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e953407-ddc7-4913-950e-9e4048297038-serviceca\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.245742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.245568    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68323c15-590d-448b-a746-58f284493bcb-host-run-k8s-cni-cncf-io\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.246794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.246774    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4888ca2e-c893-417a-b98d-ac0cd5048fa9-ovn-node-metrics-cert\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.257011 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.256981    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlbtk\" (UniqueName: \"kubernetes.io/projected/5e953407-ddc7-4913-950e-9e4048297038-kube-api-access-jlbtk\") pod \"node-ca-p4g9v\" (UID: \"5e953407-ddc7-4913-950e-9e4048297038\") " pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.258336 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.258313    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26pnf\" (UniqueName: \"kubernetes.io/projected/0297b33b-d269-40d6-8cd8-dfb44e340b55-kube-api-access-26pnf\") pod \"iptables-alerter-x4tg6\" (UID: \"0297b33b-d269-40d6-8cd8-dfb44e340b55\") " pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.259226 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.259204    2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vtxlx"
Jul 28 10:22:57.270587 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.270561    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7c49\" (UniqueName: \"kubernetes.io/projected/4888ca2e-c893-417a-b98d-ac0cd5048fa9-kube-api-access-j7c49\") pod \"ovnkube-node-w2pqh\" (UID: \"4888ca2e-c893-417a-b98d-ac0cd5048fa9\") " pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.272934 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.272913    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tqg9\" (UniqueName: \"kubernetes.io/projected/68323c15-590d-448b-a746-58f284493bcb-kube-api-access-8tqg9\") pod \"multus-tgzfb\" (UID: \"68323c15-590d-448b-a746-58f284493bcb\") " pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.280994 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.280975    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:22:57.281072 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.280999    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:22:57.281072 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.281009    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:57.281072 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.281070    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:57.781055052 +0000 UTC m=+3.058992206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:57.285732 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.285697    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hrhk\" (UniqueName: \"kubernetes.io/projected/baeba705-ba1e-436f-8c46-e08479f06175-kube-api-access-9hrhk\") pod \"multus-additional-cni-plugins-cj4zz\" (UID: \"baeba705-ba1e-436f-8c46-e08479f06175\") " pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.339872 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339850    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-run\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.339962 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339884    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpzm4\" (UniqueName: \"kubernetes.io/projected/68b3ffc0-350b-432f-8ec6-d036e68e6148-kube-api-access-vpzm4\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.339962 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339905    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.339962 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339922    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-registration-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.339962 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339946    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-hosts-file\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.339999    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-run\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340008    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-hosts-file\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340011    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340023    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-registration-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340119    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-lib-modules\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340161 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340149    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-tmp\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340168    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340185    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fms6w\" (UniqueName: \"kubernetes.io/projected/f5dc3097-a4c6-4167-b237-9533efadbc76-kube-api-access-fms6w\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340241    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-lib-modules\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340252    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brjhp\" (UniqueName: \"kubernetes.io/projected/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kube-api-access-brjhp\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340283    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340285    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-modprobe-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340325    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340354    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-host\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340368    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-modprobe-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340379    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/603776a9-6511-4a16-9f19-0b33fa8c6772-agent-certs\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340402    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-tmp-dir\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340414    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-host\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340428    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-device-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.340437    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:57.340463 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340452    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/603776a9-6511-4a16-9f19-0b33fa8c6772-konnectivity-ca\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340479    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6lwb\" (UniqueName: \"kubernetes.io/projected/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-kube-api-access-l6lwb\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340501    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-device-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340503    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.340518    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:57.840498718 +0000 UTC m=+3.118435889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340553    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-kubernetes\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340611    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-kubelet-config\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.340566    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340644    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysconfig\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.340672    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:57.840654392 +0000 UTC m=+3.118591564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340696    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-conf\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340707    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysconfig\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340728    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-var-lib-kubelet\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340746    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-tmp-dir\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340764    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-sys\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340788    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-var-lib-kubelet\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.341462 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340826    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-sys\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340823    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-sys-fs\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340868    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-tuned\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340874    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-sys-fs\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340886    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-kubernetes\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340897    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-socket-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340922    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-conf\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.340940    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-kubelet-config\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341000    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-socket-dir\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341001    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/603776a9-6511-4a16-9f19-0b33fa8c6772-konnectivity-ca\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341050    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-dbus\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341070    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-systemd\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341087    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341136    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-systemd\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341205    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-sysctl-d\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.342318 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.341206    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cefb3d3-e521-4c48-a063-77eb91f8ef91-dbus\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.343080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.342852    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/603776a9-6511-4a16-9f19-0b33fa8c6772-agent-certs\") pod \"konnectivity-agent-gh8sh\" (UID: \"603776a9-6511-4a16-9f19-0b33fa8c6772\") " pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.343080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.343029    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-tmp\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.343290 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.343271    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68b3ffc0-350b-432f-8ec6-d036e68e6148-etc-tuned\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.369744 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.369719    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brjhp\" (UniqueName: \"kubernetes.io/projected/cab68729-73f8-40f0-8ab8-8b9fec0c2a69-kube-api-access-brjhp\") pod \"aws-ebs-csi-driver-node-6wqkz\" (UID: \"cab68729-73f8-40f0-8ab8-8b9fec0c2a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.380070 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.380050    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6lwb\" (UniqueName: \"kubernetes.io/projected/ac56291e-dfd0-4c80-b2d8-f246abe4fa4b-kube-api-access-l6lwb\") pod \"node-resolver-bwn8m\" (UID: \"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b\") " pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.389600 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.389576    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fms6w\" (UniqueName: \"kubernetes.io/projected/f5dc3097-a4c6-4167-b237-9533efadbc76-kube-api-access-fms6w\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.390136 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.390115    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpzm4\" (UniqueName: \"kubernetes.io/projected/68b3ffc0-350b-432f-8ec6-d036e68e6148-kube-api-access-vpzm4\") pod \"tuned-fhq8q\" (UID: \"68b3ffc0-350b-432f-8ec6-d036e68e6148\") " pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.427947 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.427925    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:22:57.434952 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.434934    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p4g9v"
Jul 28 10:22:57.446951 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.446927    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cj4zz"
Jul 28 10:22:57.450783 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.450764    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgzfb"
Jul 28 10:22:57.457894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.457873    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x4tg6"
Jul 28 10:22:57.465585 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.465549    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwn8m"
Jul 28 10:22:57.472315 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.472289    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz"
Jul 28 10:22:57.480543 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.480387    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhq8q"
Jul 28 10:22:57.485206 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.485189    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:22:57.845791 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.845695    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:57.845791 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.845767    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:57.845800    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845878    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845903    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845918    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845918    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845882    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845974    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:58.845954608 +0000 UTC m=+4.123891765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.845993    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:58.845983644 +0000 UTC m=+4.123920802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:57.846059 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:57.846008    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:22:58.8460002 +0000 UTC m=+4.123937359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:58.263449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.263405    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 10:17:57 +0000 UTC" deadline="2028-01-04 14:20:21.395127239 +0000 UTC"
Jul 28 10:22:58.263449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.263448    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12603h57m23.131685074s"
Jul 28 10:22:58.264043 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.263501    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:58.264043 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.263637    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:22:58.338435 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.338409    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b3ffc0_350b_432f_8ec6_d036e68e6148.slice/crio-82d755c6cc293d3a792718e591df3945a1cc14fa012a0a7effd64df7b3877207 WatchSource:0}: Error finding container 82d755c6cc293d3a792718e591df3945a1cc14fa012a0a7effd64df7b3877207: Status 404 returned error can't find the container with id 82d755c6cc293d3a792718e591df3945a1cc14fa012a0a7effd64df7b3877207
Jul 28 10:22:58.342898 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.342573    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4888ca2e_c893_417a_b98d_ac0cd5048fa9.slice/crio-12b4801c445deb101118ec0ab37774052aa4d0f3e744a7d541327aa79be0eaa6 WatchSource:0}: Error finding container 12b4801c445deb101118ec0ab37774052aa4d0f3e744a7d541327aa79be0eaa6: Status 404 returned error can't find the container with id 12b4801c445deb101118ec0ab37774052aa4d0f3e744a7d541327aa79be0eaa6
Jul 28 10:22:58.343990 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.343885    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaeba705_ba1e_436f_8c46_e08479f06175.slice/crio-14be13294f54b46a709dddb320775547bcafe268089a9fa87ac1648f111de4c1 WatchSource:0}: Error finding container 14be13294f54b46a709dddb320775547bcafe268089a9fa87ac1648f111de4c1: Status 404 returned error can't find the container with id 14be13294f54b46a709dddb320775547bcafe268089a9fa87ac1648f111de4c1
Jul 28 10:22:58.347076 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.347031    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac56291e_dfd0_4c80_b2d8_f246abe4fa4b.slice/crio-65f5596171044f9688659ef89db4555dc34c78707183e52ddcbb082071514294 WatchSource:0}: Error finding container 65f5596171044f9688659ef89db4555dc34c78707183e52ddcbb082071514294: Status 404 returned error can't find the container with id 65f5596171044f9688659ef89db4555dc34c78707183e52ddcbb082071514294
Jul 28 10:22:58.349251 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.348425    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab68729_73f8_40f0_8ab8_8b9fec0c2a69.slice/crio-dd9fa9072698eda2bcd94707ded1ac6371382506893708e68d6ae6a756fc7bbd WatchSource:0}: Error finding container dd9fa9072698eda2bcd94707ded1ac6371382506893708e68d6ae6a756fc7bbd: Status 404 returned error can't find the container with id dd9fa9072698eda2bcd94707ded1ac6371382506893708e68d6ae6a756fc7bbd
Jul 28 10:22:58.350078 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.349437    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68323c15_590d_448b_a746_58f284493bcb.slice/crio-e402846f1feeb5738321c981269127706112ea8212f3b4cc7308e541bb2e9fde WatchSource:0}: Error finding container e402846f1feeb5738321c981269127706112ea8212f3b4cc7308e541bb2e9fde: Status 404 returned error can't find the container with id e402846f1feeb5738321c981269127706112ea8212f3b4cc7308e541bb2e9fde
Jul 28 10:22:58.350374 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.350177    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e953407_ddc7_4913_950e_9e4048297038.slice/crio-430d066bbfce5aef0a84105e9633c74141408f06a2dadaeab84a5b9686c874dc WatchSource:0}: Error finding container 430d066bbfce5aef0a84105e9633c74141408f06a2dadaeab84a5b9686c874dc: Status 404 returned error can't find the container with id 430d066bbfce5aef0a84105e9633c74141408f06a2dadaeab84a5b9686c874dc
Jul 28 10:22:58.351115 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:22:58.351087    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603776a9_6511_4a16_9f19_0b33fa8c6772.slice/crio-7d0fdff2521404b7a070c0cab1b1928ff903c67751f41f0f4eacf262e7a77e46 WatchSource:0}: Error finding container 7d0fdff2521404b7a070c0cab1b1928ff903c67751f41f0f4eacf262e7a77e46: Status 404 returned error can't find the container with id 7d0fdff2521404b7a070c0cab1b1928ff903c67751f41f0f4eacf262e7a77e46
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.854667    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.854997    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:58.855052    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855221    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855242    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855258    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855325    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:00.855304747 +0000 UTC m=+6.133241927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855766    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855817    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:00.855800768 +0000 UTC m=+6.133737928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855870    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:58.855934 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:58.855901    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:00.855890961 +0000 UTC m=+6.133828130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:22:59.266691 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.266627    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 10:17:57 +0000 UTC" deadline="2028-01-12 19:01:18.38408551 +0000 UTC"
Jul 28 10:22:59.266691 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.266690    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12800h38m19.117400677s"
Jul 28 10:22:59.267215 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.266785    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:22:59.267215 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:59.266925    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:22:59.267215 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.267034    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:22:59.267215 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:22:59.267145    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:22:59.292856 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.292817    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x4tg6" event={"ID":"0297b33b-d269-40d6-8cd8-dfb44e340b55","Type":"ContainerStarted","Data":"d1cf1df8b166e2cce0c8f80f3d17ca5b66069730dad3e8e8be6b31b90f19a10f"}
Jul 28 10:22:59.309301 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.309269    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhq8q" event={"ID":"68b3ffc0-350b-432f-8ec6-d036e68e6148","Type":"ContainerStarted","Data":"82d755c6cc293d3a792718e591df3945a1cc14fa012a0a7effd64df7b3877207"}
Jul 28 10:22:59.338429 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.338293    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal" event={"ID":"85547fd26d62ca170d8549b389f25e88","Type":"ContainerStarted","Data":"e0911f5c6cbcb115168a8f861c3b459e74f489a5aaf6f74a44b0d0030161aee1"}
Jul 28 10:22:59.350897 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.350700    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwn8m" event={"ID":"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b","Type":"ContainerStarted","Data":"65f5596171044f9688659ef89db4555dc34c78707183e52ddcbb082071514294"}
Jul 28 10:22:59.362234 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.362204    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerStarted","Data":"14be13294f54b46a709dddb320775547bcafe268089a9fa87ac1648f111de4c1"}
Jul 28 10:22:59.374406 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.374378    2544 generic.go:356] "Generic (PLEG): container finished" podID="bf5d45a3473f9607ff7c933ffd8b119c" containerID="91864790bc73a7662da8edd19b3a51ef68112bb6dc93cb8194fed3216a61e8ea" exitCode=0
Jul 28 10:22:59.374583 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.374464    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal" event={"ID":"bf5d45a3473f9607ff7c933ffd8b119c","Type":"ContainerDied","Data":"91864790bc73a7662da8edd19b3a51ef68112bb6dc93cb8194fed3216a61e8ea"}
Jul 28 10:22:59.387832 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.387795    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gh8sh" event={"ID":"603776a9-6511-4a16-9f19-0b33fa8c6772","Type":"ContainerStarted","Data":"7d0fdff2521404b7a070c0cab1b1928ff903c67751f41f0f4eacf262e7a77e46"}
Jul 28 10:22:59.399320 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.399266    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-31.ec2.internal" podStartSLOduration=3.399249556 podStartE2EDuration="3.399249556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:22:59.366103013 +0000 UTC m=+4.644040190" watchObservedRunningTime="2026-07-28 10:22:59.399249556 +0000 UTC m=+4.677186723"
Jul 28 10:22:59.411403 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.411318    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgzfb" event={"ID":"68323c15-590d-448b-a746-58f284493bcb","Type":"ContainerStarted","Data":"e402846f1feeb5738321c981269127706112ea8212f3b4cc7308e541bb2e9fde"}
Jul 28 10:22:59.429494 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.429435    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz" event={"ID":"cab68729-73f8-40f0-8ab8-8b9fec0c2a69","Type":"ContainerStarted","Data":"dd9fa9072698eda2bcd94707ded1ac6371382506893708e68d6ae6a756fc7bbd"}
Jul 28 10:22:59.438387 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.438333    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p4g9v" event={"ID":"5e953407-ddc7-4913-950e-9e4048297038","Type":"ContainerStarted","Data":"430d066bbfce5aef0a84105e9633c74141408f06a2dadaeab84a5b9686c874dc"}
Jul 28 10:22:59.454123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:22:59.454062    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"12b4801c445deb101118ec0ab37774052aa4d0f3e744a7d541327aa79be0eaa6"}
Jul 28 10:23:00.263938 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:00.263903    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:00.264147 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.264038    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:00.480789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:00.480755    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal" event={"ID":"bf5d45a3473f9607ff7c933ffd8b119c","Type":"ContainerStarted","Data":"5e6e04df4a315a86abd10523e9d08f9e2b749574f343ab4ae8808566ed74d4a3"}
Jul 28 10:23:00.872889 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:00.872849    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:00.873099 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:00.872938    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:00.873099 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:00.872983    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:00.873225 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873105    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:00.873225 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873169    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:04.87315089 +0000 UTC m=+10.151088050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:00.873660 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873601    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:23:00.873660 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873654    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:23:00.873958 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873668    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:00.873958 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873719    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:04.873702444 +0000 UTC m=+10.151639652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:00.873958 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873793    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:00.873958 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:00.873832    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:04.873818993 +0000 UTC m=+10.151756152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:01.266314 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:01.266231    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:01.266474 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:01.266373    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:01.266801 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:01.266784    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:01.267036 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:01.266871    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:02.264404 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:02.263874    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:02.264404 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:02.264046    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:03.266210 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:03.266160    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:03.266689 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:03.266302    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:03.266689 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:03.266680    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:03.266807 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:03.266774    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:04.263847 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:04.263795    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:04.264061 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.263971    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:04.908110    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:04.908147    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:04.908170    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908271    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908284    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908294    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908341    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:12.908323359 +0000 UTC m=+18.186260535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908423    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908449    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:12.908441385 +0000 UTC m=+18.186378537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908483    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:04.908822 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:04.908502    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:12.908495925 +0000 UTC m=+18.186433077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:05.264716 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:05.264642    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:05.264847 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:05.264730    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:05.264847 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:05.264780    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:05.264951 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:05.264866    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:06.263734 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:06.263701    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:06.264233 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:06.263825    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:07.264145 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:07.264111    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:07.264610 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:07.264442    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:07.264610 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:07.264505    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:07.264739 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:07.264621    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:08.264274 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:08.263881    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:08.264274 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:08.263988    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:09.263517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:09.263487    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:09.263676 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:09.263615    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:09.263676 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:09.263668    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:09.263760 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:09.263748    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:10.263860 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:10.263825    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:10.264324 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:10.263932    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:11.263723 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:11.263688    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:11.263888 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:11.263861    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:11.264257 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:11.264223    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:11.264328 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:11.264312    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:12.264293 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:12.264263    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:12.264747 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.264371    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:12.967568 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:12.967539    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:12.967736 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:12.967602    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:12.967736 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:12.967648    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:12.967736 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967683    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967743    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967751    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:28.967732524 +0000 UTC m=+34.245669694 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967761    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967779    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967788    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967789    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:28.967774187 +0000 UTC m=+34.245711346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:12.967880 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:12.967819    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:28.96780996 +0000 UTC m=+34.245747135 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:13.263608 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:13.263517    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:13.263761 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:13.263691    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:13.263761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:13.263745    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:13.263870 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:13.263846    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:14.264085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:14.264050    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:14.264470 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:14.264177    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:15.264578 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:15.264537    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:15.265028 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:15.264666    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:15.265028 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:15.264724    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:15.265028 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:15.264842    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:16.263342 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:16.263306    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:16.263518 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:16.263425    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:17.265608 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.265032    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:17.265608 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:17.265345    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:17.265608 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.265455    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:17.265608 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:17.265590    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:17.517071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.516902    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgzfb" event={"ID":"68323c15-590d-448b-a746-58f284493bcb","Type":"ContainerStarted","Data":"340e0912dd0d61b5ab432c521dc643268f9d63d03d58eb251c6357d5e9d3103c"}
Jul 28 10:23:17.518248 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.518221    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz" event={"ID":"cab68729-73f8-40f0-8ab8-8b9fec0c2a69","Type":"ContainerStarted","Data":"1b73e690dd40fcaee984cae776fb790b56a08db41e5fad8a85858557c57aed2f"}
Jul 28 10:23:17.519402 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.519378    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p4g9v" event={"ID":"5e953407-ddc7-4913-950e-9e4048297038","Type":"ContainerStarted","Data":"365995e990b77a773ac0015e65ebad887c2d3572e54ea81c79bc6fe1b66d55e1"}
Jul 28 10:23:17.521168 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.521150    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"fa608c2016851f17f619f7220dc9dded19115b339917e42c8188f572c8c7e284"}
Jul 28 10:23:17.521246 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.521212    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"2fa1480d6ae55e72686fa7235be2ebe576ade1e17c687d5a65e93099ae0ee7cd"}
Jul 28 10:23:17.521301 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.521263    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"7ae43fdf3dc0d9773a3b58b2a15b9ed2ee41d568e1772c5ac8c8c610818fef12"}
Jul 28 10:23:17.522427 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.522404    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhq8q" event={"ID":"68b3ffc0-350b-432f-8ec6-d036e68e6148","Type":"ContainerStarted","Data":"3b0e182540aec2d97f54c81065476c41b57b299ebffb5fbca70a4a2d29942783"}
Jul 28 10:23:17.523587 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.523552    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwn8m" event={"ID":"ac56291e-dfd0-4c80-b2d8-f246abe4fa4b","Type":"ContainerStarted","Data":"264dd87d1ad8fd8b6958a66b90977b8bfa49a76d9f49a05f1c4d77be86747ef4"}
Jul 28 10:23:17.525124 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.525100    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="d93e8b12a0ad38bcb55ba8cbdf4bb77dcfd308dc23ca38238ab666de8acceea2" exitCode=0
Jul 28 10:23:17.525208 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.525174    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"d93e8b12a0ad38bcb55ba8cbdf4bb77dcfd308dc23ca38238ab666de8acceea2"}
Jul 28 10:23:17.526893 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.526873    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gh8sh" event={"ID":"603776a9-6511-4a16-9f19-0b33fa8c6772","Type":"ContainerStarted","Data":"2ea0e811194857e70731373b33d7d547a2b5e61a87a826cfeee4df0babbc5d43"}
Jul 28 10:23:17.548689 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.548651    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-31.ec2.internal" podStartSLOduration=21.54863993 podStartE2EDuration="21.54863993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:00.506751345 +0000 UTC m=+5.784688523" watchObservedRunningTime="2026-07-28 10:23:17.54863993 +0000 UTC m=+22.826577128"
Jul 28 10:23:17.597008 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.596965    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tgzfb" podStartSLOduration=3.8357600979999997 podStartE2EDuration="22.596952009s" totalImagesPullingTime="18.761191911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:17.548891518 +0000 UTC m=+22.826828692" watchObservedRunningTime="2026-07-28 10:23:17.596952009 +0000 UTC m=+22.874889182"
Jul 28 10:23:17.670224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.670184    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gh8sh" podStartSLOduration=2.96929231 podStartE2EDuration="21.670169454s" totalImagesPullingTime="18.700877144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:17.669870212 +0000 UTC m=+22.947807389" watchObservedRunningTime="2026-07-28 10:23:17.670169454 +0000 UTC m=+22.948106676"
Jul 28 10:23:17.670306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.670279    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhq8q" podStartSLOduration=3.906761311 podStartE2EDuration="22.670272019s" totalImagesPullingTime="18.763510708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:17.626850592 +0000 UTC m=+22.904787766" watchObservedRunningTime="2026-07-28 10:23:17.670272019 +0000 UTC m=+22.948209194"
Jul 28 10:23:17.753516 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.753464    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bwn8m" podStartSLOduration=4.018297817 podStartE2EDuration="22.753443934s" totalImagesPullingTime="18.735146117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:17.752453073 +0000 UTC m=+23.030390272" watchObservedRunningTime="2026-07-28 10:23:17.753443934 +0000 UTC m=+23.031381108"
Jul 28 10:23:17.753653 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:17.753590    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p4g9v" podStartSLOduration=4.040906994 podStartE2EDuration="22.753574025s" totalImagesPullingTime="18.712667031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:17.712824217 +0000 UTC m=+22.990761390" watchObservedRunningTime="2026-07-28 10:23:17.753574025 +0000 UTC m=+23.031511194"
Jul 28 10:23:18.263785 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.263752    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:18.263971 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:18.263880    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:18.532988 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.532956    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"14ce6513e76a395f5cd0e74faaf6a840c94676903e9861e7d33c85d36e68cde5"}
Jul 28 10:23:18.533633 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.533048    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"c37862d53f823a2b0fcdb51c1671defe0ae083383dd7d40be5f3871150db44be"}
Jul 28 10:23:18.533633 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.533096    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"08658c6b0f97ebc04452e240fa14de5e8b52d5bd09873d23d4fce8caec261934"}
Jul 28 10:23:18.534694 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.534661    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x4tg6" event={"ID":"0297b33b-d269-40d6-8cd8-dfb44e340b55","Type":"ContainerStarted","Data":"57cace404b2921049d0fe50d6f3dc73166d52b544db1a2a584702c0741d9ba56"}
Jul 28 10:23:18.554356 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.554313    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x4tg6" podStartSLOduration=4.815784956 podStartE2EDuration="23.554299052s" totalImagesPullingTime="18.738514096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:18.554111704 +0000 UTC m=+23.832048878" watchObservedRunningTime="2026-07-28 10:23:18.554299052 +0000 UTC m=+23.832236270"
Jul 28 10:23:18.592904 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:18.592881    2544 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 10:23:19.196699 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.196595    2544 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T10:23:18.59289903Z","UUID":"e6675a10-6b3c-4cd8-bf5d-e2dd8b01fe0c","Handler":null,"Name":"","Endpoint":""}
Jul 28 10:23:19.198246 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.198205    2544 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 10:23:19.198376 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.198254    2544 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 10:23:19.263771 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.263746    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:19.263972 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:19.263888    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:19.263972 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.263951    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:19.264090 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:19.264057    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:19.538686 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:19.538619    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz" event={"ID":"cab68729-73f8-40f0-8ab8-8b9fec0c2a69","Type":"ContainerStarted","Data":"94218f328efc8025496a1fdf08368da8ed8e56d157e2a7ec2ff8594fa438c474"}
Jul 28 10:23:20.263706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:20.263464    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:20.263913 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:20.263737    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:20.543960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:20.543875    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz" event={"ID":"cab68729-73f8-40f0-8ab8-8b9fec0c2a69","Type":"ContainerStarted","Data":"d210d4060e9b7cd4b0a40b64f9a35ab914c3ab850a8671e48587c26ba32b4428"}
Jul 28 10:23:20.547912 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:20.547884    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"1de4a6295d791cfcc889720c39b89f410fc3d20725e99e8ad6bfd3f3b70509de"}
Jul 28 10:23:21.264025 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:21.263993    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:21.264235 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:21.264002    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:21.264235 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:21.264125    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:21.264235 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:21.264204    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:22.264465 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.264311    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:22.264921 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:22.264610    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:22.458521 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.458491    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:23:22.459150 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.459131    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:23:22.476283 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.476222    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wqkz" podStartSLOduration=6.590813887 podStartE2EDuration="27.476204907s" totalImagesPullingTime="20.88539102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:20.561862687 +0000 UTC m=+25.839799861" watchObservedRunningTime="2026-07-28 10:23:22.476204907 +0000 UTC m=+27.754142081"
Jul 28 10:23:22.555369 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.555336    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" event={"ID":"4888ca2e-c893-417a-b98d-ac0cd5048fa9","Type":"ContainerStarted","Data":"e60357e54bb100687c3663d8bfc2edfac569555dd564c1ac22a9d2bd71eb78a5"}
Jul 28 10:23:22.555615 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.555597    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:23:22.557208 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.557184    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="8ed27d570c8e6e93049afb803c2a3c3735a0e32e65a7ae5acde759b04de0eebc" exitCode=0
Jul 28 10:23:22.557281 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.557249    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"8ed27d570c8e6e93049afb803c2a3c3735a0e32e65a7ae5acde759b04de0eebc"}
Jul 28 10:23:22.557444 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.557430    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:23:22.557797 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.557782    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gh8sh"
Jul 28 10:23:22.571972 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.571952    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:23:22.587458 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:22.587422    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh" podStartSLOduration=8.787858102 podStartE2EDuration="27.587408821s" totalImagesPullingTime="18.799550719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:22.586057783 +0000 UTC m=+27.863994958" watchObservedRunningTime="2026-07-28 10:23:22.587408821 +0000 UTC m=+27.865345994"
Jul 28 10:23:23.263720 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:23.263680    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:23.263909 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:23.263682    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:23.263909 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:23.263804    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:23.263909 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:23.263866    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:23.560334 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:23.560269    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:23:23.560726 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:23.560351    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:23:23.574874 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:23.574853    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:23:24.263884 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.263829    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:24.264029 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:24.263922    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:24.563825 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.563748    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="2d8cf1bc6b5be2be422e3604ac5c36a3d4757576ed3d87a90ba214141d17477c" exitCode=0
Jul 28 10:23:24.564401 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.563844    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"2d8cf1bc6b5be2be422e3604ac5c36a3d4757576ed3d87a90ba214141d17477c"}
Jul 28 10:23:24.690796 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.690769    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-67ksr"]
Jul 28 10:23:24.690970 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.690894    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:24.691030 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:24.690988    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:24.691476 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.691460    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzjkp"]
Jul 28 10:23:24.691569 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.691536    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:24.691692 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:24.691671    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:24.692173 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.692099    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9h54r"]
Jul 28 10:23:24.692322 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:24.692268    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:24.692396 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:24.692377    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:26.263794 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:26.263617    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:26.264091 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:26.263653    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:26.264091 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:26.263869    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:26.264091 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:26.263653    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:26.264091 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:26.263948    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:26.264091 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:26.264029    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:26.570211 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:26.570136    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="5cdc1380d6c065d53aebe4b8b1cad1b2cf33262447bf8e89423f62fae430b771" exitCode=0
Jul 28 10:23:26.570338 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:26.570210    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"5cdc1380d6c065d53aebe4b8b1cad1b2cf33262447bf8e89423f62fae430b771"}
Jul 28 10:23:28.264248 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.264216    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:28.264860 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.264272    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:28.264860 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.264282    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:28.264860 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.264341    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-67ksr" podUID="0cefb3d3-e521-4c48-a063-77eb91f8ef91"
Jul 28 10:23:28.264860 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.264472    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9h54r" podUID="f5dc3097-a4c6-4167-b237-9533efadbc76"
Jul 28 10:23:28.264860 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.264549    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzjkp" podUID="33a4f66e-b565-4ca2-9af7-a52c48515807"
Jul 28 10:23:28.990874 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.990842    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:28.990874 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.990892    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:28.991122 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:28.990917    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:28.991122 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991017    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:28.991122 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991061    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret podName:0cefb3d3-e521-4c48-a063-77eb91f8ef91 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:00.991049394 +0000 UTC m=+66.268986548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret") pod "global-pull-secret-syncer-67ksr" (UID: "0cefb3d3-e521-4c48-a063-77eb91f8ef91") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:23:28.991122 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991019    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:28.991280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991132    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs podName:f5dc3097-a4c6-4167-b237-9533efadbc76 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:00.991112231 +0000 UTC m=+66.269049399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs") pod "network-metrics-daemon-9h54r" (UID: "f5dc3097-a4c6-4167-b237-9533efadbc76") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:23:28.991280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991065    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:23:28.991280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991168    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:23:28.991280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991182    2544 projected.go:196] Error preparing data for projected volume kube-api-access-9czq8 for pod openshift-network-diagnostics/network-check-target-mzjkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:28.991280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:28.991221    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8 podName:33a4f66e-b565-4ca2-9af7-a52c48515807 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:00.991209286 +0000 UTC m=+66.269146455 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9czq8" (UniqueName: "kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8") pod "network-check-target-mzjkp" (UID: "33a4f66e-b565-4ca2-9af7-a52c48515807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:23:29.985663 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:29.985624    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-31.ec2.internal" event="NodeReady"
Jul 28 10:23:29.986233 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:29.985758    2544 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 10:23:30.042096 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.042066    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:23:30.070517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.070490    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"]
Jul 28 10:23:30.070671 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.070654    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.073389 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.073006    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 10:23:30.073389 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.073033    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f7477\""
Jul 28 10:23:30.073389 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.073253    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 10:23:30.073656 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.073580    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 10:23:30.086474 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.086450    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"]
Jul 28 10:23:30.086604 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.086589    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.092224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.092194    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 10:23:30.092343 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.092238    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.092343 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.092260    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.092533 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.092477    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 10:23:30.092665 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.092578    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pvzjs\""
Jul 28 10:23:30.094057 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.094040    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 10:23:30.104389 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.104363    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"]
Jul 28 10:23:30.104523 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.104506    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.111713 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.111477    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.111713 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.111496    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fmpxk\""
Jul 28 10:23:30.111713 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.111542    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 10:23:30.111856 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.111834    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.126451 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.126434    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"]
Jul 28 10:23:30.126603 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.126586    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.132020 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.131493    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.132020 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.131688    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.132020 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.131876    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rqwpv\""
Jul 28 10:23:30.132863 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.132844    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 10:23:30.132984 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.132961    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 10:23:30.146602 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.146584    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"]
Jul 28 10:23:30.146703 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.146683    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.149613 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.149593    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 10:23:30.149760 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.149723    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.150057 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.150039    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-w9wg6\""
Jul 28 10:23:30.150148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.150039    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.150148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.150113    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 10:23:30.168803 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168781    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"]
Jul 28 10:23:30.168919 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168823    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"]
Jul 28 10:23:30.168919 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168862    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:23:30.168919 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168883    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"]
Jul 28 10:23:30.168919 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168902    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bmpbj"]
Jul 28 10:23:30.169208 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.168994    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"
Jul 28 10:23:30.171368 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.171351    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.171517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.171359    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.171846 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.171827    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7pvl2\""
Jul 28 10:23:30.192800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.192776    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9d49bfb5-n7bxk"]
Jul 28 10:23:30.192918 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.192902    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.196158 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.196138    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.197185 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.197161    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 10:23:30.197441 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.197420    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 10:23:30.197940 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.197716    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pv8wd\""
Jul 28 10:23:30.198073 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.198055    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.198875 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.198854    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjt54\" (UniqueName: \"kubernetes.io/projected/c17c2d08-5d58-4f09-be1c-07b53349c486-kube-api-access-rjt54\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.198976 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.198889    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.198976 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.198914    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.198988    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/662b8125-a009-4bad-ba14-574df1776a3e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.199085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199015    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5vfw\" (UniqueName: \"kubernetes.io/projected/662b8125-a009-4bad-ba14-574df1776a3e-kube-api-access-w5vfw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.199085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199052    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199085 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199077    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199276 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199107    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp4mc\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199276 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199141    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199276 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199218    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/662b8125-a009-4bad-ba14-574df1776a3e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.199276 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199242    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.199276 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199269    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.199485 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.199295    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.205205 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.205151    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:30.208261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.208240    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 10:23:30.208859 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.208841    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-gg27m"]
Jul 28 10:23:30.208966 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.208952    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.212532 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212513    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.212637 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212549    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 10:23:30.212826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212810    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.212929 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212843    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 10:23:30.212929 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212809    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 10:23:30.213034 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.212979    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z4hk5\""
Jul 28 10:23:30.213399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.213381    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 10:23:30.231171 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.231138    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"]
Jul 28 10:23:30.231281 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.231265    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.234049 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.234032    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.234147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.234071    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 10:23:30.235339 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.235321    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8tsx8\""
Jul 28 10:23:30.235450 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.235370    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 10:23:30.235636 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.235622    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.245626 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.245608    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 10:23:30.249012 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.248991    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"]
Jul 28 10:23:30.249126 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.249107    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.251813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.251798    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 10:23:30.251923 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.251797    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pkkt4\""
Jul 28 10:23:30.252466 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.252441    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 10:23:30.271374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271349    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"]
Jul 28 10:23:30.271475 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271389    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"]
Jul 28 10:23:30.271475 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271426    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:23:30.271475 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271451    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.271680 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271491    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:23:30.271918 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.271859    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:23:30.274377 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.274358    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.274687 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.274644    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 10:23:30.274865 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.274849    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.275421 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275404    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 10:23:30.275522 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275434    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mqqnj\""
Jul 28 10:23:30.275633 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275617    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 10:23:30.275772 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275758    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 10:23:30.275863 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275849    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 10:23:30.275863 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275858    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 10:23:30.275968 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.275892    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.276069 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.276049    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 10:23:30.276159 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.276117    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.276251 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.276237    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k8src\""
Jul 28 10:23:30.285994 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.285973    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"]
Jul 28 10:23:30.286125 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.286108    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"
Jul 28 10:23:30.291210 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.291193    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6744v\""
Jul 28 10:23:30.295496 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.295478    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"]
Jul 28 10:23:30.295631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.295612    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.298497 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.298480    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 10:23:30.299668 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299634    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp4mc\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.299761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299697    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-stats-auth\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.299761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299739    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.299761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299758    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.299901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299784    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/daebe161-b184-41a5-9bb6-e41424e14559-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.299901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299809    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmkbf\" (UniqueName: \"kubernetes.io/projected/daebe161-b184-41a5-9bb6-e41424e14559-kube-api-access-dmkbf\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.299901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299832    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7vqw\" (UniqueName: \"kubernetes.io/projected/79027776-4d57-4fac-a422-0a2cce20c33f-kube-api-access-x7vqw\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.299901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299855    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t2jz\" (UniqueName: \"kubernetes.io/projected/435525f2-d692-4697-b299-0a6484bbd281-kube-api-access-6t2jz\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.299901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299874    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299906    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299922    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-default-certificate\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299945    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-serving-cert\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.299999    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79027776-4d57-4fac-a422-0a2cce20c33f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300019    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79027776-4d57-4fac-a422-0a2cce20c33f-config\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300044    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300062    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/662b8125-a009-4bad-ba14-574df1776a3e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300084    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5vfw\" (UniqueName: \"kubernetes.io/projected/662b8125-a009-4bad-ba14-574df1776a3e-kube-api-access-w5vfw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300102    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-config\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.300131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300127    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrrrb\" (UniqueName: \"kubernetes.io/projected/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-kube-api-access-zrrrb\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.300599 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.300159    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:30.300599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300171    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.300599 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.300177    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:30.300599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.300386    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.300599 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.300436    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:23:30.800421764 +0000 UTC m=+36.078358916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:30.301162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301145    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.301641 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301623    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw28j\" (UniqueName: \"kubernetes.io/projected/eae859ae-347d-4ba4-89c0-19470bfa944b-kube-api-access-tw28j\") pod \"volume-data-source-validator-549c6d54cf-fd7cx\" (UID: \"eae859ae-347d-4ba4-89c0-19470bfa944b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"
Jul 28 10:23:30.301732 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301687    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.301784 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.301767    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:30.301825 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.301810    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:30.801797345 +0000 UTC m=+36.079734518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:30.301891 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301872    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/662b8125-a009-4bad-ba14-574df1776a3e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.301982 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301899    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.301982 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.301923    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-trusted-ca\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.302522 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.302505    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.302624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.302544    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjt54\" (UniqueName: \"kubernetes.io/projected/c17c2d08-5d58-4f09-be1c-07b53349c486-kube-api-access-rjt54\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.302624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.302589    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.302726 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.302421    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/662b8125-a009-4bad-ba14-574df1776a3e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.302938 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.302898    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.303252 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.303224    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.305256 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.305211    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/662b8125-a009-4bad-ba14-574df1776a3e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.305350 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.305330    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.305407 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.305348    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.312647 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.312537    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.312955 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.312937    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5vfw\" (UniqueName: \"kubernetes.io/projected/662b8125-a009-4bad-ba14-574df1776a3e-kube-api-access-w5vfw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl\" (UID: \"662b8125-a009-4bad-ba14-574df1776a3e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.313380 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.313362    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp4mc\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.316595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.316548    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjt54\" (UniqueName: \"kubernetes.io/projected/c17c2d08-5d58-4f09-be1c-07b53349c486-kube-api-access-rjt54\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.317779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317760    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9d49bfb5-n7bxk"]
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317797    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"]
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317813    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"]
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317832    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-gg27m"]
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317852    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bmpbj"]
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317861    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.317885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317872    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"]
Jul 28 10:23:30.318207 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317897    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"]
Jul 28 10:23:30.318207 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317918    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"]
Jul 28 10:23:30.318207 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.317935    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"]
Jul 28 10:23:30.321392 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.321374    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 10:23:30.321680 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.321657    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7bvgt\""
Jul 28 10:23:30.323916 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.323897    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xkrf6"]
Jul 28 10:23:30.341437 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.341413    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xkrf6"]
Jul 28 10:23:30.341533 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.341498    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.345799 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.345780    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 10:23:30.346278 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.346253    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 10:23:30.346862 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.346838    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8f5cv\""
Jul 28 10:23:30.367197 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.367174    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tgdp6"]
Jul 28 10:23:30.380615 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.380588    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:30.386266 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.386248    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.388143 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.388122    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tgdp6"]
Jul 28 10:23:30.388880 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.388861    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 10:23:30.388975 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.388910    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 10:23:30.389364 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.389349    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 10:23:30.389960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.389942    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9dkdj\""
Jul 28 10:23:30.397931 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.397911    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"
Jul 28 10:23:30.403729 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403694    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s57ts\" (UniqueName: \"kubernetes.io/projected/b0a78cf4-c930-49c8-b56b-7cda3559fd47-kube-api-access-s57ts\") pod \"network-check-source-7d88cb55d8-p6nrr\" (UID: \"b0a78cf4-c930-49c8-b56b-7cda3559fd47\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"
Jul 28 10:23:30.403814 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403751    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-trusted-ca\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.403814 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403770    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-serving-cert\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.403894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403818    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.403894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403853    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-tmp\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.403894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403873    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.404004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403917    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-service-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.404004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403942    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384146b8-be47-4971-bce7-f3de889994b3-tmp\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.404004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403961    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.404004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.403978    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a38fb5c-56b2-400f-9840-7f7acd2e20be-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.404147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404005    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-stats-auth\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.404147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404032    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhm5m\" (UniqueName: \"kubernetes.io/projected/4a38fb5c-56b2-400f-9840-7f7acd2e20be-kube-api-access-qhm5m\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.404147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404059    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.404147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404082    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9977dae4-55ad-4243-b0af-1fa676491f00-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.404147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404134    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-snapshots\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.404294 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404180    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk47p\" (UniqueName: \"kubernetes.io/projected/9977dae4-55ad-4243-b0af-1fa676491f00-kube-api-access-hk47p\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.404294 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.404188    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:30.404294 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404220    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/daebe161-b184-41a5-9bb6-e41424e14559-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.404294 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.404260    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:30.904240354 +0000 UTC m=+36.182177511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:30.404294 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404283    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkbf\" (UniqueName: \"kubernetes.io/projected/daebe161-b184-41a5-9bb6-e41424e14559-kube-api-access-dmkbf\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404314    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7vqw\" (UniqueName: \"kubernetes.io/projected/79027776-4d57-4fac-a422-0a2cce20c33f-kube-api-access-x7vqw\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404344    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t2jz\" (UniqueName: \"kubernetes.io/projected/435525f2-d692-4697-b299-0a6484bbd281-kube-api-access-6t2jz\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404372    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404399    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404429    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/384146b8-be47-4971-bce7-f3de889994b3-klusterlet-config\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404453    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.404503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404493    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-default-certificate\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404520    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-serving-cert\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404543    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79027776-4d57-4fac-a422-0a2cce20c33f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404586    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-trusted-ca\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.404714    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:30.904695473 +0000 UTC m=+36.182632662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404752    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79027776-4d57-4fac-a422-0a2cce20c33f-config\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404785    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/daebe161-b184-41a5-9bb6-e41424e14559-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404786    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404830    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-config\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.404847    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:30.404867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404848    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4sw\" (UniqueName: \"kubernetes.io/projected/384146b8-be47-4971-bce7-f3de889994b3-kube-api-access-hr4sw\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.405335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404880    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ltp6\" (UniqueName: \"kubernetes.io/projected/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-kube-api-access-4ltp6\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.405335 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.404898    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:30.904886677 +0000 UTC m=+36.182823831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:30.405335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404916    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrrrb\" (UniqueName: \"kubernetes.io/projected/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-kube-api-access-zrrrb\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.405335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404933    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb033949-9ce9-4b56-a7e9-7f43383d5af3-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.405335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.404962    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.405789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.405764    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-config\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.405907 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.405888    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79027776-4d57-4fac-a422-0a2cce20c33f-config\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.406000 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.405983    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw28j\" (UniqueName: \"kubernetes.io/projected/eae859ae-347d-4ba4-89c0-19470bfa944b-kube-api-access-tw28j\") pod \"volume-data-source-validator-549c6d54cf-fd7cx\" (UID: \"eae859ae-347d-4ba4-89c0-19470bfa944b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"
Jul 28 10:23:30.406703 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.406661    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-stats-auth\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.406839 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.406818    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79027776-4d57-4fac-a422-0a2cce20c33f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.407650 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.407628    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-default-certificate\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.408654 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.408626    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-serving-cert\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.417129 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.417105    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmkbf\" (UniqueName: \"kubernetes.io/projected/daebe161-b184-41a5-9bb6-e41424e14559-kube-api-access-dmkbf\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.417450 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.417430    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrrrb\" (UniqueName: \"kubernetes.io/projected/b5ddfb23-d3c5-412f-a084-79ece8f6ac37-kube-api-access-zrrrb\") pod \"console-operator-584dcb6b54-bmpbj\" (UID: \"b5ddfb23-d3c5-412f-a084-79ece8f6ac37\") " pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.418748 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.418728    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7vqw\" (UniqueName: \"kubernetes.io/projected/79027776-4d57-4fac-a422-0a2cce20c33f-kube-api-access-x7vqw\") pod \"service-ca-operator-5d6c75f84f-8gnbl\" (UID: \"79027776-4d57-4fac-a422-0a2cce20c33f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.419795 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.419771    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw28j\" (UniqueName: \"kubernetes.io/projected/eae859ae-347d-4ba4-89c0-19470bfa944b-kube-api-access-tw28j\") pod \"volume-data-source-validator-549c6d54cf-fd7cx\" (UID: \"eae859ae-347d-4ba4-89c0-19470bfa944b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"
Jul 28 10:23:30.419883 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.419820    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t2jz\" (UniqueName: \"kubernetes.io/projected/435525f2-d692-4697-b299-0a6484bbd281-kube-api-access-6t2jz\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.456891 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.456862    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"
Jul 28 10:23:30.473164 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.473137    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:30.479573 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.479542    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"
Jul 28 10:23:30.506659 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506635    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:30.506846 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506821    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-service-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.506936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506863    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.506936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506889    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384146b8-be47-4971-bce7-f3de889994b3-tmp\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.507027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506942    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.506976    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a38fb5c-56b2-400f-9840-7f7acd2e20be-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.507027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507004    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2378764-8651-4edf-b937-f2738f9f2236-cert\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.507138 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507034    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhm5m\" (UniqueName: \"kubernetes.io/projected/4a38fb5c-56b2-400f-9840-7f7acd2e20be-kube-api-access-qhm5m\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.507138 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507075    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9977dae4-55ad-4243-b0af-1fa676491f00-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507138 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507104    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-snapshots\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507138 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507126    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2cmd\" (UniqueName: \"kubernetes.io/projected/95553483-b6f2-47e4-a9dd-02266be7a816-kube-api-access-q2cmd\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.507286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507153    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk47p\" (UniqueName: \"kubernetes.io/projected/9977dae4-55ad-4243-b0af-1fa676491f00-kube-api-access-hk47p\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507187    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507229    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/384146b8-be47-4971-bce7-f3de889994b3-klusterlet-config\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.507286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507253    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507313    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4sw\" (UniqueName: \"kubernetes.io/projected/384146b8-be47-4971-bce7-f3de889994b3-kube-api-access-hr4sw\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507340    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ltp6\" (UniqueName: \"kubernetes.io/projected/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-kube-api-access-4ltp6\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507370    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb033949-9ce9-4b56-a7e9-7f43383d5af3-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507398    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507412    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-service-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507449 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507432    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95553483-b6f2-47e4-a9dd-02266be7a816-config-volume\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507486    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s57ts\" (UniqueName: \"kubernetes.io/projected/b0a78cf4-c930-49c8-b56b-7cda3559fd47-kube-api-access-s57ts\") pod \"network-check-source-7d88cb55d8-p6nrr\" (UID: \"b0a78cf4-c930-49c8-b56b-7cda3559fd47\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507530    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-serving-cert\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507577    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507607    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-tmp\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507633    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.507679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507657    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc8w6\" (UniqueName: \"kubernetes.io/projected/c2378764-8651-4edf-b937-f2738f9f2236-kube-api-access-bc8w6\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.508008 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507687    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95553483-b6f2-47e4-a9dd-02266be7a816-tmp-dir\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.508008 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507855    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9977dae4-55ad-4243-b0af-1fa676491f00-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.508272 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.507252    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/384146b8-be47-4971-bce7-f3de889994b3-tmp\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.508698    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.509057    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-snapshots\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.509298    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.509307    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-tmp\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.509354    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.009336058 +0000 UTC m=+36.287273231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:30.509512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.509357    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb033949-9ce9-4b56-a7e9-7f43383d5af3-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:30.510883 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.510855    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a38fb5c-56b2-400f-9840-7f7acd2e20be-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.511145 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.511118    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.511992 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.511950    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.512291 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.512270    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/384146b8-be47-4971-bce7-f3de889994b3-klusterlet-config\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.512532 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.512512    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.512628 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.512602    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-serving-cert\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.522729 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.522713    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4sw\" (UniqueName: \"kubernetes.io/projected/384146b8-be47-4971-bce7-f3de889994b3-kube-api-access-hr4sw\") pod \"klusterlet-addon-workmgr-65588577c6-ht2rm\" (UID: \"384146b8-be47-4971-bce7-f3de889994b3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.523967 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.523948    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhm5m\" (UniqueName: \"kubernetes.io/projected/4a38fb5c-56b2-400f-9840-7f7acd2e20be-kube-api-access-qhm5m\") pod \"managed-serviceaccount-addon-agent-5db76f6f8b-smw55\" (UID: \"4a38fb5c-56b2-400f-9840-7f7acd2e20be\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.526795 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.526778    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ltp6\" (UniqueName: \"kubernetes.io/projected/3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122-kube-api-access-4ltp6\") pod \"insights-operator-5885fdb759-gg27m\" (UID: \"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122\") " pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.526964 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.526947    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s57ts\" (UniqueName: \"kubernetes.io/projected/b0a78cf4-c930-49c8-b56b-7cda3559fd47-kube-api-access-s57ts\") pod \"network-check-source-7d88cb55d8-p6nrr\" (UID: \"b0a78cf4-c930-49c8-b56b-7cda3559fd47\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"
Jul 28 10:23:30.527266 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.527249    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9977dae4-55ad-4243-b0af-1fa676491f00-ca\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.529188 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.529167    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk47p\" (UniqueName: \"kubernetes.io/projected/9977dae4-55ad-4243-b0af-1fa676491f00-kube-api-access-hk47p\") pod \"cluster-proxy-proxy-agent-78cd48d4d8-27lpv\" (UID: \"9977dae4-55ad-4243-b0af-1fa676491f00\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.542211 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.542190    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-gg27m"
Jul 28 10:23:30.594905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.594875    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:23:30.608880 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.608858    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2cmd\" (UniqueName: \"kubernetes.io/projected/95553483-b6f2-47e4-a9dd-02266be7a816-kube-api-access-q2cmd\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.609000 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.608983    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95553483-b6f2-47e4-a9dd-02266be7a816-config-volume\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.609086 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609072    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc8w6\" (UniqueName: \"kubernetes.io/projected/c2378764-8651-4edf-b937-f2738f9f2236-kube-api-access-bc8w6\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.609123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609103    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95553483-b6f2-47e4-a9dd-02266be7a816-tmp-dir\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.609163 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609137    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.609201 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609174    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2378764-8651-4edf-b937-f2738f9f2236-cert\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.609328 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.609305    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:30.609426 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.609366    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.109347504 +0000 UTC m=+36.387284681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:30.609493 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609459    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95553483-b6f2-47e4-a9dd-02266be7a816-tmp-dir\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.609549 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.609507    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95553483-b6f2-47e4-a9dd-02266be7a816-config-volume\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.612317 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.612298    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2378764-8651-4edf-b937-f2738f9f2236-cert\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.619744 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.619718    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc8w6\" (UniqueName: \"kubernetes.io/projected/c2378764-8651-4edf-b937-f2738f9f2236-kube-api-access-bc8w6\") pod \"ingress-canary-tgdp6\" (UID: \"c2378764-8651-4edf-b937-f2738f9f2236\") " pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.620199 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.620177    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2cmd\" (UniqueName: \"kubernetes.io/projected/95553483-b6f2-47e4-a9dd-02266be7a816-kube-api-access-q2cmd\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:30.645392 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.645359    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"
Jul 28 10:23:30.653570 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.653523    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:30.673397 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.673379    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"
Jul 28 10:23:30.696801 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.696782    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tgdp6"
Jul 28 10:23:30.810212 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.810148    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:30.810331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.810267    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:30.810331 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.810319    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:30.810405 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.810367    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:30.810405 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.810377    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:30.810405 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.810380    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.81036432 +0000 UTC m=+37.088301472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:30.810497 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.810419    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.810408651 +0000 UTC m=+37.088345803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:30.918886 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.914137    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:30.918886 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.914203    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.918886 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.914263    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:30.918886 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.914441    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:30.918886 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.914503    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.914484058 +0000 UTC m=+37.192421224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:30.920677 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.920622    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.920599135 +0000 UTC m=+37.198536288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:30.922873 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.922774    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:30.922873 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:30.922846    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:31.922826274 +0000 UTC m=+37.200763428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:30.930851 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:30.929332    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:31.017246 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.015264    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:31.017246 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.015661    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:31.017246 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.015723    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:32.015706561 +0000 UTC m=+37.293643732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:31.030220 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.030163    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:31.117475 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.116715    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:31.117475 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.116945    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:31.117475 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.117009    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:32.116990281 +0000 UTC m=+37.394927456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:31.203634 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.203602    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"]
Jul 28 10:23:31.211745 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.211386    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"]
Jul 28 10:23:31.219994 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:31.219313    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9977dae4_55ad_4243_b0af_1fa676491f00.slice/crio-b68886b78b79802bfa131180fde0d3becb06a0463abd9aba2679548ab478d5c6 WatchSource:0}: Error finding container b68886b78b79802bfa131180fde0d3becb06a0463abd9aba2679548ab478d5c6: Status 404 returned error can't find the container with id b68886b78b79802bfa131180fde0d3becb06a0463abd9aba2679548ab478d5c6
Jul 28 10:23:31.219994 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:31.219663    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384146b8_be47_4971_bce7_f3de889994b3.slice/crio-9b9e75565c6bcdcf22ee14b979089e701d51aa1e1d832882f48e2c85db002e67 WatchSource:0}: Error finding container 9b9e75565c6bcdcf22ee14b979089e701d51aa1e1d832882f48e2c85db002e67: Status 404 returned error can't find the container with id 9b9e75565c6bcdcf22ee14b979089e701d51aa1e1d832882f48e2c85db002e67
Jul 28 10:23:31.243551 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.243519    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-gg27m"]
Jul 28 10:23:31.247215 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:31.247185    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b788e7c_ecd7_4aa5_bfcb_0a1ad1d97122.slice/crio-87895cf019cc559ba0c8de6c205f48309829be1d0b875c4fca84369a31f27986 WatchSource:0}: Error finding container 87895cf019cc559ba0c8de6c205f48309829be1d0b875c4fca84369a31f27986: Status 404 returned error can't find the container with id 87895cf019cc559ba0c8de6c205f48309829be1d0b875c4fca84369a31f27986
Jul 28 10:23:31.253520 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.253498    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55"]
Jul 28 10:23:31.258730 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.258697    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tgdp6"]
Jul 28 10:23:31.261484 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.261463    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr"]
Jul 28 10:23:31.272675 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:31.272593    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a78cf4_c930_49c8_b56b_7cda3559fd47.slice/crio-fef51c09d96bbee60dffc2d60df2d3539a84ead8a7baa92db21b0538d5702086 WatchSource:0}: Error finding container fef51c09d96bbee60dffc2d60df2d3539a84ead8a7baa92db21b0538d5702086: Status 404 returned error can't find the container with id fef51c09d96bbee60dffc2d60df2d3539a84ead8a7baa92db21b0538d5702086
Jul 28 10:23:31.273026 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:31.272832    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae859ae_347d_4ba4_89c0_19470bfa944b.slice/crio-1162883b2551e01c0269ab41e26127a7ae2c2a6d3dab3926fd0d386109a5e07b WatchSource:0}: Error finding container 1162883b2551e01c0269ab41e26127a7ae2c2a6d3dab3926fd0d386109a5e07b: Status 404 returned error can't find the container with id 1162883b2551e01c0269ab41e26127a7ae2c2a6d3dab3926fd0d386109a5e07b
Jul 28 10:23:31.276595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.274920    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx"]
Jul 28 10:23:31.276595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.274969    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl"]
Jul 28 10:23:31.276595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.274988    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl"]
Jul 28 10:23:31.276595 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.275015    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bmpbj"]
Jul 28 10:23:31.336122 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.336061    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:31.431366 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.431330    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:31.583258 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.583231    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx" event={"ID":"eae859ae-347d-4ba4-89c0-19470bfa944b","Type":"ContainerStarted","Data":"1162883b2551e01c0269ab41e26127a7ae2c2a6d3dab3926fd0d386109a5e07b"}
Jul 28 10:23:31.584256 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.584236    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-gg27m" event={"ID":"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122","Type":"ContainerStarted","Data":"87895cf019cc559ba0c8de6c205f48309829be1d0b875c4fca84369a31f27986"}
Jul 28 10:23:31.585152 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.585135    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr" event={"ID":"b0a78cf4-c930-49c8-b56b-7cda3559fd47","Type":"ContainerStarted","Data":"fef51c09d96bbee60dffc2d60df2d3539a84ead8a7baa92db21b0538d5702086"}
Jul 28 10:23:31.587694 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.587644    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="8ccf14140a1dbcd317e943afff4050bf3789c65ad9012cc1b3765c39d10f2a86" exitCode=0
Jul 28 10:23:31.587763 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.587710    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"8ccf14140a1dbcd317e943afff4050bf3789c65ad9012cc1b3765c39d10f2a86"}
Jul 28 10:23:31.588764 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.588739    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl" event={"ID":"662b8125-a009-4bad-ba14-574df1776a3e","Type":"ContainerStarted","Data":"02f4e575c5b7c183d31c5c3805596b5ee732d80e7907c7c7335091458a0b2850"}
Jul 28 10:23:31.589754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.589737    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tgdp6" event={"ID":"c2378764-8651-4edf-b937-f2738f9f2236","Type":"ContainerStarted","Data":"93a52763ea3624fa7dd9262dd1d33b4e63189174482acac78cbf138401e1ea5c"}
Jul 28 10:23:31.590779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.590763    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm" event={"ID":"384146b8-be47-4971-bce7-f3de889994b3","Type":"ContainerStarted","Data":"9b9e75565c6bcdcf22ee14b979089e701d51aa1e1d832882f48e2c85db002e67"}
Jul 28 10:23:31.591896 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.591861    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerStarted","Data":"b68886b78b79802bfa131180fde0d3becb06a0463abd9aba2679548ab478d5c6"}
Jul 28 10:23:31.593063 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.593040    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" event={"ID":"b5ddfb23-d3c5-412f-a084-79ece8f6ac37","Type":"ContainerStarted","Data":"34c995651e5a6e6313fdf52034d9434654348fdef05af13c3069a22e31a7e6f2"}
Jul 28 10:23:31.594126 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.594071    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55" event={"ID":"4a38fb5c-56b2-400f-9840-7f7acd2e20be","Type":"ContainerStarted","Data":"815263fcfeb18f3951bcffcdfc8831a940f5dfa23493a75c669a55a100157840"}
Jul 28 10:23:31.595106 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.595085    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl" event={"ID":"79027776-4d57-4fac-a422-0a2cce20c33f","Type":"ContainerStarted","Data":"3ce4e5082f3a37334d375ee35876414f1f1ff0cdc506895324c5c6cb0f809fd1"}
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.824776    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.824982    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.825117    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.825131    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.825179    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.825188    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:23:33.825170661 +0000 UTC m=+39.103107813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:31.825526 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.825217    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:33.82520634 +0000 UTC m=+39.103143511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.926331    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.926500    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:31.926544    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.926720    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:33.926701204 +0000 UTC m=+39.204638361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.926811    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.926844    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:33.926833874 +0000 UTC m=+39.204771030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.926899    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:31.927261 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:31.926931    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:33.926919954 +0000 UTC m=+39.204857113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:32.028885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.028045    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:32.028885 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:32.028434    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:32.028885 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:32.028497    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:34.028479078 +0000 UTC m=+39.306416252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:32.123106 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.123057    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:32.129716 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.129123    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:32.129716 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:32.129285    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:32.129716 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:32.129340    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:34.129323211 +0000 UTC m=+39.407260369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:32.226500 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.226413    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:32.580202 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.580123    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:32.631349 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.631313    2544 generic.go:356] "Generic (PLEG): container finished" podID="baeba705-ba1e-436f-8c46-e08479f06175" containerID="5d6705f14204d9ceacba3f5d129b3fe7659c52919ff61a1281b4796609d0b760" exitCode=0
Jul 28 10:23:32.631582 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.631395    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerDied","Data":"5d6705f14204d9ceacba3f5d129b3fe7659c52919ff61a1281b4796609d0b760"}
Jul 28 10:23:32.677101 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:32.677065    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:33.671973 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.671661    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" event={"ID":"baeba705-ba1e-436f-8c46-e08479f06175","Type":"ContainerStarted","Data":"cbadcd50829e3d3da802c66c0077aaae5f837ce269ff398966cc491b440a2fc3"}
Jul 28 10:23:33.730923 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.730487    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cj4zz" podStartSLOduration=8.62604449 podStartE2EDuration="38.730466803s" totalImagesPullingTime="30.104422313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:33.729649702 +0000 UTC m=+39.007586877" watchObservedRunningTime="2026-07-28 10:23:33.730466803 +0000 UTC m=+39.008403978"
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.849420    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.849512    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.850531    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.850549    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.850625    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:23:37.850606832 +0000 UTC m=+43.128544002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.850816    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:33.851222 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.850855    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:37.850841561 +0000 UTC m=+43.128778725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:33.858890 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.858808    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:33.951618 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.951519    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:33.951618 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.951603    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:33.951846 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.951668    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:33.953169 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.952204    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:33.953169 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.952281    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:37.952259711 +0000 UTC m=+43.230196867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:33.953169 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.952762    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:37.952746566 +0000 UTC m=+43.230683723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:33.954037 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.953955    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:33.954037 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:33.954013    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:37.953997207 +0000 UTC m=+43.231934360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:33.955454 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:33.955314    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:34.052624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.052280    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:34.052624 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:34.052450    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:34.052624 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:34.052512    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:38.052493778 +0000 UTC m=+43.330430935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:34.153679 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.153648    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:34.153910 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:34.153893    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:34.153998 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:34.153953    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:38.153934881 +0000 UTC m=+43.431872041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:34.459492 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.459374    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:34.559991 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.559871    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:34.679411 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.679275    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx" event={"ID":"eae859ae-347d-4ba4-89c0-19470bfa944b","Type":"ContainerStarted","Data":"64bb3968aa711ab9bfa4daa5a2607e1671cad78204c86037538ff2387f46a6cc"}
Jul 28 10:23:34.797958 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.797921    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:34.879685 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:34.879639    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:37.399659 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.399621    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:37.457696 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.457658    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:37.892750 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.892717    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:37.892933 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.892774    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:37.892933 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.892889    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:37.892933 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.892903    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:37.892933 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.892920    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:37.893128 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.892950    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.892932178 +0000 UTC m=+51.170869354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:37.893128 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.893051    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.893032017 +0000 UTC m=+51.170969169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:37.993499 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.993460    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:37.993702 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.993527    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:37.993702 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:37.993595    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:37.994453 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.993826    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:37.994453 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.993892    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.993870858 +0000 UTC m=+51.271808014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:37.994453 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.994302    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:37.994453 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.994348    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.994334063 +0000 UTC m=+51.272271222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:37.994453 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:37.994422    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.994412225 +0000 UTC m=+51.272349382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:38.094526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:38.094488    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:38.094783 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:38.094763    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:38.094916 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:38.094829    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:46.094810395 +0000 UTC m=+51.372747554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:38.196160 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:38.196082    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:38.196289 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:38.196218    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:38.196289 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:38.196277    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:46.196264001 +0000 UTC m=+51.474201156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:38.421687 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:38.421638    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:38.482363 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:38.482286    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:43.704298 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.704265    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl" event={"ID":"662b8125-a009-4bad-ba14-574df1776a3e","Type":"ContainerStarted","Data":"4fb79a0addb68d617053167ab3b6e6c1f0dfb36c1816ab5a06e3fb77dbee421a"}
Jul 28 10:23:43.706422 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.706395    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tgdp6" event={"ID":"c2378764-8651-4edf-b937-f2738f9f2236","Type":"ContainerStarted","Data":"3c3756c0cdbe294dfd39354092ea0adb1350ae89dd6f42846758cf95c4e70a98"}
Jul 28 10:23:43.709102 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.709078    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerStarted","Data":"156e2085f3a3ff50a2d0ffacfb4008e04a3e4209dfaa5e9375cf0866ba46ba47"}
Jul 28 10:23:43.709198 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.709116    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerStarted","Data":"152da14cc71e8c0a32c49836240adfb52c328bbea4eca6df210d8b344e2855bb"}
Jul 28 10:23:43.709198 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.709133    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerStarted","Data":"b3b9d1ddb009adc75e397b792bedb404ba99eb204a736192f24363680f3a3609"}
Jul 28 10:23:43.710769 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.710749    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/0.log"
Jul 28 10:23:43.710861 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.710783    2544 generic.go:356] "Generic (PLEG): container finished" podID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37" containerID="cbcaebda7e46ed65ee095f4ae8ee766184a62a6230b8347a4d13564f109ae718" exitCode=255
Jul 28 10:23:43.710861 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.710848    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" event={"ID":"b5ddfb23-d3c5-412f-a084-79ece8f6ac37","Type":"ContainerDied","Data":"cbcaebda7e46ed65ee095f4ae8ee766184a62a6230b8347a4d13564f109ae718"}
Jul 28 10:23:43.711103 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.711088    2544 scope.go:117] "RemoveContainer" containerID="cbcaebda7e46ed65ee095f4ae8ee766184a62a6230b8347a4d13564f109ae718"
Jul 28 10:23:43.713816 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.713792    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55" event={"ID":"4a38fb5c-56b2-400f-9840-7f7acd2e20be","Type":"ContainerStarted","Data":"b969867704f8d09f512287b79bd2c7e59c1236f2d0cd8ac15fce39b0caed85c8"}
Jul 28 10:23:43.716083 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.715725    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl" event={"ID":"79027776-4d57-4fac-a422-0a2cce20c33f","Type":"ContainerStarted","Data":"82cd4bc7b4fe2fe9e52e68cba2dcbefba86441802ab6ee62d7360138ee263b52"}
Jul 28 10:23:43.718516 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.718489    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-gg27m" event={"ID":"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122","Type":"ContainerStarted","Data":"0be2f5ae97feb1af78735462f3f4407952b70a27d139f27d1bf54b44b98e60f0"}
Jul 28 10:23:43.721134 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.720794    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr" event={"ID":"b0a78cf4-c930-49c8-b56b-7cda3559fd47","Type":"ContainerStarted","Data":"b7587fb43fb4344b85cea9b566e76287c1426634a46f985f29842eb718fa0f26"}
Jul 28 10:23:43.722587 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.722519    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fd7cx" podStartSLOduration=16.945212058 podStartE2EDuration="19.722501822s" totalImagesPullingTime="2.777289764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:34.698812997 +0000 UTC m=+39.976750171" watchObservedRunningTime="2026-07-28 10:23:43.722501822 +0000 UTC m=+49.000438997"
Jul 28 10:23:43.723150 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.723121    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl" podStartSLOduration=7.8194923339999995 podStartE2EDuration="19.723113152s" totalImagesPullingTime="11.903620818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.721576882 +0000 UTC m=+48.999514059" watchObservedRunningTime="2026-07-28 10:23:43.723113152 +0000 UTC m=+49.001050309"
Jul 28 10:23:43.740967 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.740514    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tgdp6" podStartSLOduration=1.826877412 podStartE2EDuration="13.740496209s" totalImagesPullingTime="11.913618797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.740493508 +0000 UTC m=+49.018430683" watchObservedRunningTime="2026-07-28 10:23:43.740496209 +0000 UTC m=+49.018433389"
Jul 28 10:23:43.758784 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.758743    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl" podStartSLOduration=7.95015448 podStartE2EDuration="19.758726616s" totalImagesPullingTime="11.808572136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.757764522 +0000 UTC m=+49.035701697" watchObservedRunningTime="2026-07-28 10:23:43.758726616 +0000 UTC m=+49.036663788"
Jul 28 10:23:43.773698 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.772482    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p6nrr" podStartSLOduration=7.8691043050000005 podStartE2EDuration="19.772466591s" totalImagesPullingTime="11.903362286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.771813301 +0000 UTC m=+49.049750476" watchObservedRunningTime="2026-07-28 10:23:43.772466591 +0000 UTC m=+49.050403765"
Jul 28 10:23:43.804619 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:43.804575    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-gg27m" podStartSLOduration=7.918209963 podStartE2EDuration="19.804540762s" totalImagesPullingTime="11.886330799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.804066075 +0000 UTC m=+49.082003252" watchObservedRunningTime="2026-07-28 10:23:43.804540762 +0000 UTC m=+49.082477938"
Jul 28 10:23:44.513726 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.513667    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db76f6f8b-smw55" podStartSLOduration=7.585177676 podStartE2EDuration="19.513645213s" totalImagesPullingTime="11.928467537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:43.821843659 +0000 UTC m=+49.099780836" watchObservedRunningTime="2026-07-28 10:23:44.513645213 +0000 UTC m=+49.791582387"
Jul 28 10:23:44.514246 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.514226    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nsnq9"]
Jul 28 10:23:44.527532 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.527502    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nsnq9"]
Jul 28 10:23:44.527676 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.527647    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.530416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.530195    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 10:23:44.530416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.530220    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 10:23:44.530605 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.530479    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kn9mk\""
Jul 28 10:23:44.546026 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.546000    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:44.639683 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.639625    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:44.650695 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.650670    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-crio-socket\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.650826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.650726    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.650826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.650758    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.650826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.650793    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-data-volume\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.651188 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.650899    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnvd\" (UniqueName: \"kubernetes.io/projected/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-api-access-dxnvd\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.676227 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.676187    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"]
Jul 28 10:23:44.692892 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.692864    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"
Jul 28 10:23:44.694530 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.694506    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"]
Jul 28 10:23:44.695004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.694987    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b6rz7\""
Jul 28 10:23:44.695458 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.695433    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 10:23:44.695553 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.695433    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 10:23:44.726442 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.726419    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:44.728156 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.728140    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/1.log"
Jul 28 10:23:44.728545 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.728529    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/0.log"
Jul 28 10:23:44.728630 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.728580    2544 generic.go:356] "Generic (PLEG): container finished" podID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f" exitCode=255
Jul 28 10:23:44.728716 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.728686    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" event={"ID":"b5ddfb23-d3c5-412f-a084-79ece8f6ac37","Type":"ContainerDied","Data":"268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"}
Jul 28 10:23:44.728771 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.728749    2544 scope.go:117] "RemoveContainer" containerID="cbcaebda7e46ed65ee095f4ae8ee766184a62a6230b8347a4d13564f109ae718"
Jul 28 10:23:44.729761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.729745    2544 scope.go:117] "RemoveContainer" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"
Jul 28 10:23:44.729897 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:44.729874    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bmpbj_openshift-console-operator(b5ddfb23-d3c5-412f-a084-79ece8f6ac37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podUID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37"
Jul 28 10:23:44.749278 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.749216    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" podStartSLOduration=7.791697942 podStartE2EDuration="19.749203108s" totalImagesPullingTime="11.957505166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:44.748379693 +0000 UTC m=+50.026316870" watchObservedRunningTime="2026-07-28 10:23:44.749203108 +0000 UTC m=+50.027140281"
Jul 28 10:23:44.751610 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751586    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-crio-socket\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.751697 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751634    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-crio-socket\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.751697 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751668    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.751813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751720    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.751813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751751    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-data-volume\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.751813 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:44.751753    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:23:44.751813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751786    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99r8h\" (UniqueName: \"kubernetes.io/projected/8da11606-2cc2-4ae0-82b1-5372c89aa961-kube-api-access-99r8h\") pod \"migrator-6865c9967-9hhr6\" (UID: \"8da11606-2cc2-4ae0-82b1-5372c89aa961\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"
Jul 28 10:23:44.751813 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:44.751810    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.251791859 +0000 UTC m=+50.529729016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : configmap "kube-rbac-proxy" not found
Jul 28 10:23:44.752043 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.751868    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnvd\" (UniqueName: \"kubernetes.io/projected/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-api-access-dxnvd\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.752093 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.752078    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-data-volume\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.752220 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:44.752203    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:44.752274 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:44.752253    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:45.252238793 +0000 UTC m=+50.530175961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:44.763662 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.763640    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnvd\" (UniqueName: \"kubernetes.io/projected/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-api-access-dxnvd\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:44.854783 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.853453    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99r8h\" (UniqueName: \"kubernetes.io/projected/8da11606-2cc2-4ae0-82b1-5372c89aa961-kube-api-access-99r8h\") pod \"migrator-6865c9967-9hhr6\" (UID: \"8da11606-2cc2-4ae0-82b1-5372c89aa961\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"
Jul 28 10:23:44.864537 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.864504    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99r8h\" (UniqueName: \"kubernetes.io/projected/8da11606-2cc2-4ae0-82b1-5372c89aa961-kube-api-access-99r8h\") pod \"migrator-6865c9967-9hhr6\" (UID: \"8da11606-2cc2-4ae0-82b1-5372c89aa961\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"
Jul 28 10:23:44.875733 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.875712    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:44.977613 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:44.977550    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:45.003507 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.003486    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"
Jul 28 10:23:45.076887 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.076860    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:45.155456 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.155429    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6"]
Jul 28 10:23:45.159013 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:45.158992    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da11606_2cc2_4ae0_82b1_5372c89aa961.slice/crio-ea79db85f0caf6167a3b8b7408dc6502cc72f1e90ebe10bf3991128cd113163a WatchSource:0}: Error finding container ea79db85f0caf6167a3b8b7408dc6502cc72f1e90ebe10bf3991128cd113163a: Status 404 returned error can't find the container with id ea79db85f0caf6167a3b8b7408dc6502cc72f1e90ebe10bf3991128cd113163a
Jul 28 10:23:45.256536 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.256510    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:45.256659 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.256549    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:45.256659 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.256618    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:23:45.256755 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.256678    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:46.256663306 +0000 UTC m=+51.534600476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : configmap "kube-rbac-proxy" not found
Jul 28 10:23:45.256755 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.256714    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:45.256831 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.256762    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:46.256750498 +0000 UTC m=+51.534687650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:45.410167 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.410090    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:45.464081 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.464056    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:45.510852 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.510831    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwn8m_ac56291e-dfd0-4c80-b2d8-f246abe4fa4b/dns-node-resolver/0.log"
Jul 28 10:23:45.708897 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.708815    2544 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log" limit=1024
Jul 28 10:23:45.732775 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.732740    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6" event={"ID":"8da11606-2cc2-4ae0-82b1-5372c89aa961","Type":"ContainerStarted","Data":"ea79db85f0caf6167a3b8b7408dc6502cc72f1e90ebe10bf3991128cd113163a"}
Jul 28 10:23:45.734531 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.734507    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/1.log"
Jul 28 10:23:45.734942 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.734920    2544 scope.go:117] "RemoveContainer" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"
Jul 28 10:23:45.735121 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.735104    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bmpbj_openshift-console-operator(b5ddfb23-d3c5-412f-a084-79ece8f6ac37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podUID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37"
Jul 28 10:23:45.963254 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.963165    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:23:45.963254 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:45.963248    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:23:45.963471 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.963450    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:23:45.963527 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.963496    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:23:45.963527 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.963513    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7577b96c84-t5gcn: secret "image-registry-tls" not found
Jul 28 10:23:45.963527 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.963520    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls podName:c17c2d08-5d58-4f09-be1c-07b53349c486 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:01.963501052 +0000 UTC m=+67.241438226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls") pod "cluster-samples-operator-6748489656-bjq8m" (UID: "c17c2d08-5d58-4f09-be1c-07b53349c486") : secret "samples-operator-tls" not found
Jul 28 10:23:45.963723 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:45.963546    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls podName:048d224f-dacc-4fc0-ae66-9d750de55b6b nodeName:}" failed. No retries permitted until 2026-07-28 10:24:01.963535953 +0000 UTC m=+67.241473109 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls") pod "image-registry-7577b96c84-t5gcn" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b") : secret "image-registry-tls" not found
Jul 28 10:23:46.064725 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.064690    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:46.064883 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.064754    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:23:46.064883 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.064841    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:02.064820217 +0000 UTC m=+67.342757373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : configmap references non-existent config key: service-ca.crt
Jul 28 10:23:46.065011 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.064913    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:23:46.065011 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.064957    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs podName:435525f2-d692-4697-b299-0a6484bbd281 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:02.064941592 +0000 UTC m=+67.342878751 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs") pod "router-default-9d49bfb5-n7bxk" (UID: "435525f2-d692-4697-b299-0a6484bbd281") : secret "router-metrics-certs-default" not found
Jul 28 10:23:46.065127 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.065013    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:23:46.065127 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.065118    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:46.065241 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.065160    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls podName:daebe161-b184-41a5-9bb6-e41424e14559 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:02.065149658 +0000 UTC m=+67.343086812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dd6jz" (UID: "daebe161-b184-41a5-9bb6-e41424e14559") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:23:46.166251 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.166213    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:23:46.166536 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.166520    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 10:23:46.166608 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.166602    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert podName:bb033949-9ce9-4b56-a7e9-7f43383d5af3 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:02.166588915 +0000 UTC m=+67.444526067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pnm92" (UID: "bb033949-9ce9-4b56-a7e9-7f43383d5af3") : secret "networking-console-plugin-cert" not found
Jul 28 10:23:46.267347 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.267265    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:23:46.267493 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.267359    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:46.267493 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.267393    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:46.267633 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.267521    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:46.267633 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.267612    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:48.267591256 +0000 UTC m=+53.545528432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:46.268025 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.268008    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:23:46.268076 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:46.268064    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls podName:95553483-b6f2-47e4-a9dd-02266be7a816 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:02.26804805 +0000 UTC m=+67.545985218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls") pod "dns-default-xkrf6" (UID: "95553483-b6f2-47e4-a9dd-02266be7a816") : secret "dns-default-metrics-tls" not found
Jul 28 10:23:46.268643 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.268622    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:46.911540 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:46.911514    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p4g9v_5e953407-ddc7-4913-950e-9e4048297038/node-ca/0.log"
Jul 28 10:23:47.295691 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.295617    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-zsbq5"]
Jul 28 10:23:47.310706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.310675    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-zsbq5"]
Jul 28 10:23:47.310936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.310834    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.313849 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.313826    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 10:23:47.314809 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.314787    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 10:23:47.314907 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.314807    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 10:23:47.314957 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.314871    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 10:23:47.314957 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.314883    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mrqr4\""
Jul 28 10:23:47.316738 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.316719    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tgdp6_c2378764-8651-4edf-b937-f2738f9f2236/serve-healthcheck-canary/0.log"
Jul 28 10:23:47.380004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.379971    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cad8bad6-611b-4461-b2b2-7701e1205581-signing-cabundle\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.380146 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.380044    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cad8bad6-611b-4461-b2b2-7701e1205581-signing-key\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.380234 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.380220    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9jwm\" (UniqueName: \"kubernetes.io/projected/cad8bad6-611b-4461-b2b2-7701e1205581-kube-api-access-s9jwm\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.480812 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.480776    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9jwm\" (UniqueName: \"kubernetes.io/projected/cad8bad6-611b-4461-b2b2-7701e1205581-kube-api-access-s9jwm\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.480985 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.480851    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cad8bad6-611b-4461-b2b2-7701e1205581-signing-cabundle\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.480985 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.480888    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cad8bad6-611b-4461-b2b2-7701e1205581-signing-key\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.481708 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.481683    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cad8bad6-611b-4461-b2b2-7701e1205581-signing-cabundle\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.483687 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.483665    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cad8bad6-611b-4461-b2b2-7701e1205581-signing-key\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.490426 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.490408    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9jwm\" (UniqueName: \"kubernetes.io/projected/cad8bad6-611b-4461-b2b2-7701e1205581-kube-api-access-s9jwm\") pod \"service-ca-658d7b7c78-zsbq5\" (UID: \"cad8bad6-611b-4461-b2b2-7701e1205581\") " pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.623158 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.623121    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5"
Jul 28 10:23:47.744312 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.744273    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6" event={"ID":"8da11606-2cc2-4ae0-82b1-5372c89aa961","Type":"ContainerStarted","Data":"de98ff479ca75f023a3457886ce13576bec826e56b88c5dde29dc0acb109dd0f"}
Jul 28 10:23:47.744469 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:47.744342    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6" event={"ID":"8da11606-2cc2-4ae0-82b1-5372c89aa961","Type":"ContainerStarted","Data":"8fea71574c87bcd19ec447522e0b83e6bdd7b53fc05d82077736631ec6f7eb51"}
Jul 28 10:23:48.289454 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.289418    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:48.289964 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:48.289577    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:48.289964 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:48.289641    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:23:52.289623818 +0000 UTC m=+57.567560969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:48.501778 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.501735    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-9hhr6" podStartSLOduration=2.798176014 podStartE2EDuration="4.501715973s" totalImagesPullingTime="1.703539959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:47.763408759 +0000 UTC m=+53.041345937" watchObservedRunningTime="2026-07-28 10:23:48.501715973 +0000 UTC m=+53.779653183"
Jul 28 10:23:48.501941 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.501928    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-zsbq5"]
Jul 28 10:23:48.504662 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:23:48.504626    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad8bad6_611b_4461_b2b2_7701e1205581.slice/crio-a0a11958239996ba36879d11279502f9e05cde1bb5dd6134766c44761629aadc WatchSource:0}: Error finding container a0a11958239996ba36879d11279502f9e05cde1bb5dd6134766c44761629aadc: Status 404 returned error can't find the container with id a0a11958239996ba36879d11279502f9e05cde1bb5dd6134766c44761629aadc
Jul 28 10:23:48.749271 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.749230    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5" event={"ID":"cad8bad6-611b-4461-b2b2-7701e1205581","Type":"ContainerStarted","Data":"390b4eef1630790d619aeee808884390b2a362b3d08b1b11dfe990b0bcfad2fb"}
Jul 28 10:23:48.749467 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.749321    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5" event={"ID":"cad8bad6-611b-4461-b2b2-7701e1205581","Type":"ContainerStarted","Data":"a0a11958239996ba36879d11279502f9e05cde1bb5dd6134766c44761629aadc"}
Jul 28 10:23:48.750789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.750766    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm" event={"ID":"384146b8-be47-4971-bce7-f3de889994b3","Type":"ContainerStarted","Data":"76e0d64215382a7e9471a0a98ca83058b0b01ba879c570c721386a24f9badbd8"}
Jul 28 10:23:48.775163 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.775120    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-zsbq5" podStartSLOduration=1.77510857 podStartE2EDuration="1.77510857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:48.775091733 +0000 UTC m=+54.053028908" watchObservedRunningTime="2026-07-28 10:23:48.77510857 +0000 UTC m=+54.053045744"
Jul 28 10:23:48.797400 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:48.797358    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm" podStartSLOduration=6.601782972 podStartE2EDuration="23.797345761s" totalImagesPullingTime="17.195562789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:23:48.796235349 +0000 UTC m=+54.074172523" watchObservedRunningTime="2026-07-28 10:23:48.797345761 +0000 UTC m=+54.075282935"
Jul 28 10:23:49.754435 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:49.754397    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:49.756026 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:49.756006    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65588577c6-ht2rm"
Jul 28 10:23:50.507662 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:50.507629    2544 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:50.507844 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:50.507692    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:23:50.508071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:50.508055    2544 scope.go:117] "RemoveContainer" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"
Jul 28 10:23:50.508262 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:50.508240    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bmpbj_openshift-console-operator(b5ddfb23-d3c5-412f-a084-79ece8f6ac37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podUID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37"
Jul 28 10:23:52.326099 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:52.326065    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:23:52.326470 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:52.326211    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:52.326470 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:23:52.326270    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls podName:321c2faa-5209-4b2f-ae32-4e3bb51f1b38 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:00.326255793 +0000 UTC m=+65.604192946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nsnq9" (UID: "321c2faa-5209-4b2f-ae32-4e3bb51f1b38") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:23:55.576878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:23:55.576851    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w2pqh"
Jul 28 10:24:00.395742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.395711    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:24:00.398244 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.398211    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321c2faa-5209-4b2f-ae32-4e3bb51f1b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nsnq9\" (UID: \"321c2faa-5209-4b2f-ae32-4e3bb51f1b38\") " pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:24:00.442843 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.442822    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kn9mk\""
Jul 28 10:24:00.450825 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.450809    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nsnq9"
Jul 28 10:24:00.584143 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.584115    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nsnq9"]
Jul 28 10:24:00.587202 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:00.587175    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321c2faa_5209_4b2f_ae32_4e3bb51f1b38.slice/crio-5db9a881b14aabb495424b3cde200ed69f3bf153209d1b4b17e3749d05fe4861 WatchSource:0}: Error finding container 5db9a881b14aabb495424b3cde200ed69f3bf153209d1b4b17e3749d05fe4861: Status 404 returned error can't find the container with id 5db9a881b14aabb495424b3cde200ed69f3bf153209d1b4b17e3749d05fe4861
Jul 28 10:24:00.792118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:00.792048    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nsnq9" event={"ID":"321c2faa-5209-4b2f-ae32-4e3bb51f1b38","Type":"ContainerStarted","Data":"5db9a881b14aabb495424b3cde200ed69f3bf153209d1b4b17e3749d05fe4861"}
Jul 28 10:24:01.000110 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.000078    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:24:01.000280 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.000193    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:24:01.000280 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.000213    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:24:01.002707 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.002682    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9czq8\" (UniqueName: \"kubernetes.io/projected/33a4f66e-b565-4ca2-9af7-a52c48515807-kube-api-access-9czq8\") pod \"network-check-target-mzjkp\" (UID: \"33a4f66e-b565-4ca2-9af7-a52c48515807\") " pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:24:01.003226 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.003205    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 10:24:01.003306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.003284    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 10:24:01.012706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.012686    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cefb3d3-e521-4c48-a063-77eb91f8ef91-original-pull-secret\") pod \"global-pull-secret-syncer-67ksr\" (UID: \"0cefb3d3-e521-4c48-a063-77eb91f8ef91\") " pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:24:01.012800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.012728    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5dc3097-a4c6-4167-b237-9533efadbc76-metrics-certs\") pod \"network-metrics-daemon-9h54r\" (UID: \"f5dc3097-a4c6-4167-b237-9533efadbc76\") " pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:24:01.204004 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.203976    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k8src\""
Jul 28 10:24:01.212155 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.212122    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9h54r"
Jul 28 10:24:01.238149 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.238121    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-67ksr"
Jul 28 10:24:01.238615 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.238572    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mqqnj\""
Jul 28 10:24:01.246252 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.246232    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:24:01.408763 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.408687    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9h54r"]
Jul 28 10:24:01.447163 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:01.447124    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dc3097_a4c6_4167_b237_9533efadbc76.slice/crio-3e26368ed18d74f9cfab61f88436ebf028c45dbc0463df5647ace6ba02ffb1c4 WatchSource:0}: Error finding container 3e26368ed18d74f9cfab61f88436ebf028c45dbc0463df5647ace6ba02ffb1c4: Status 404 returned error can't find the container with id 3e26368ed18d74f9cfab61f88436ebf028c45dbc0463df5647ace6ba02ffb1c4
Jul 28 10:24:01.454537 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.454436    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-67ksr"]
Jul 28 10:24:01.459934 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:01.459905    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cefb3d3_e521_4c48_a063_77eb91f8ef91.slice/crio-9f5692a01e92c393b85d1bd49550da7c99d08801af9155c806b0f8abcb515d01 WatchSource:0}: Error finding container 9f5692a01e92c393b85d1bd49550da7c99d08801af9155c806b0f8abcb515d01: Status 404 returned error can't find the container with id 9f5692a01e92c393b85d1bd49550da7c99d08801af9155c806b0f8abcb515d01
Jul 28 10:24:01.503421 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.502657    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzjkp"]
Jul 28 10:24:01.506086 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:01.506058    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a4f66e_b565_4ca2_9af7_a52c48515807.slice/crio-3931bcfb8f933ddf2a544011eb55e242b5d5b26d38af6a75c09ed26cb8ef54fd WatchSource:0}: Error finding container 3931bcfb8f933ddf2a544011eb55e242b5d5b26d38af6a75c09ed26cb8ef54fd: Status 404 returned error can't find the container with id 3931bcfb8f933ddf2a544011eb55e242b5d5b26d38af6a75c09ed26cb8ef54fd
Jul 28 10:24:01.798966 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.798878    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzjkp" event={"ID":"33a4f66e-b565-4ca2-9af7-a52c48515807","Type":"ContainerStarted","Data":"7d7cc52650269f20f4a00279a119be98016b00c02d674a9e8a7801fadc0a59d6"}
Jul 28 10:24:01.798966 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.798940    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzjkp" event={"ID":"33a4f66e-b565-4ca2-9af7-a52c48515807","Type":"ContainerStarted","Data":"3931bcfb8f933ddf2a544011eb55e242b5d5b26d38af6a75c09ed26cb8ef54fd"}
Jul 28 10:24:01.799178 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.799128    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:24:01.800635 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.800585    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9h54r" event={"ID":"f5dc3097-a4c6-4167-b237-9533efadbc76","Type":"ContainerStarted","Data":"3e26368ed18d74f9cfab61f88436ebf028c45dbc0463df5647ace6ba02ffb1c4"}
Jul 28 10:24:01.802366 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.802341    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-67ksr" event={"ID":"0cefb3d3-e521-4c48-a063-77eb91f8ef91","Type":"ContainerStarted","Data":"9f5692a01e92c393b85d1bd49550da7c99d08801af9155c806b0f8abcb515d01"}
Jul 28 10:24:01.816547 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:01.816508    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mzjkp" podStartSLOduration=66.816494699 podStartE2EDuration="1m6.816494699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:01.815670079 +0000 UTC m=+67.093607254" watchObservedRunningTime="2026-07-28 10:24:01.816494699 +0000 UTC m=+67.094431876"
Jul 28 10:24:02.013189 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.013152    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:24:02.013370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.013238    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:24:02.015937 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.015911    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"image-registry-7577b96c84-t5gcn\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") " pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:24:02.016086 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.015971    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c17c2d08-5d58-4f09-be1c-07b53349c486-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bjq8m\" (UID: \"c17c2d08-5d58-4f09-be1c-07b53349c486\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:24:02.114306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.114198    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:24:02.114306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.114267    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:02.114306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.114305    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:02.115175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.115148    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/435525f2-d692-4697-b299-0a6484bbd281-service-ca-bundle\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:02.117219 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.117182    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/435525f2-d692-4697-b299-0a6484bbd281-metrics-certs\") pod \"router-default-9d49bfb5-n7bxk\" (UID: \"435525f2-d692-4697-b299-0a6484bbd281\") " pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:02.117681 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.117661    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/daebe161-b184-41a5-9bb6-e41424e14559-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dd6jz\" (UID: \"daebe161-b184-41a5-9bb6-e41424e14559\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:24:02.185264 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.185081    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f7477\""
Jul 28 10:24:02.192630 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.192593    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:24:02.214804 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.214777    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:24:02.217473 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.217414    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fmpxk\""
Jul 28 10:24:02.218517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.218467    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb033949-9ce9-4b56-a7e9-7f43383d5af3-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pnm92\" (UID: \"bb033949-9ce9-4b56-a7e9-7f43383d5af3\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:24:02.224483 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.224459    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"
Jul 28 10:24:02.239766 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.239741    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rqwpv\""
Jul 28 10:24:02.247522 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.247503    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"
Jul 28 10:24:02.316816 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.316216    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:24:02.320916 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.320654    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z4hk5\""
Jul 28 10:24:02.324228 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.324177    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95553483-b6f2-47e4-a9dd-02266be7a816-metrics-tls\") pod \"dns-default-xkrf6\" (UID: \"95553483-b6f2-47e4-a9dd-02266be7a816\") " pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:24:02.329235 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.329030    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:02.364147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.363929    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pkkt4\""
Jul 28 10:24:02.369989 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.369901    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"
Jul 28 10:24:02.404163 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.404117    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:24:02.417044 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:02.417015    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048d224f_dacc_4fc0_ae66_9d750de55b6b.slice/crio-19b47a38d8ba34665df12472163b89a7a3b5538ac477a4f0493983cb5ab3b600 WatchSource:0}: Error finding container 19b47a38d8ba34665df12472163b89a7a3b5538ac477a4f0493983cb5ab3b600: Status 404 returned error can't find the container with id 19b47a38d8ba34665df12472163b89a7a3b5538ac477a4f0493983cb5ab3b600
Jul 28 10:24:02.464737 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.464705    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m"]
Jul 28 10:24:02.482082 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.481884    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8f5cv\""
Jul 28 10:24:02.492661 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.489655    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:24:02.507265 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.507180    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz"]
Jul 28 10:24:02.516328 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:02.513524    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaebe161_b184_41a5_9bb6_e41424e14559.slice/crio-c06453b652e0620636071be5acb4f23df4c216e73602012489caa6174daa236e WatchSource:0}: Error finding container c06453b652e0620636071be5acb4f23df4c216e73602012489caa6174daa236e: Status 404 returned error can't find the container with id c06453b652e0620636071be5acb4f23df4c216e73602012489caa6174daa236e
Jul 28 10:24:02.582221 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.581798    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9d49bfb5-n7bxk"]
Jul 28 10:24:02.590330 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:02.590299    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435525f2_d692_4697_b299_0a6484bbd281.slice/crio-f9df2373a4eb28057979f59bbf152cfd011d3956a8f68bbf8b606087eebf8079 WatchSource:0}: Error finding container f9df2373a4eb28057979f59bbf152cfd011d3956a8f68bbf8b606087eebf8079: Status 404 returned error can't find the container with id f9df2373a4eb28057979f59bbf152cfd011d3956a8f68bbf8b606087eebf8079
Jul 28 10:24:02.599619 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.599596    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pnm92"]
Jul 28 10:24:02.602839 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:02.602810    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb033949_9ce9_4b56_a7e9_7f43383d5af3.slice/crio-60174eaa2bd6d81588519bfe2a3f9e112d23b08816c4798bd1e07e61ad9b112d WatchSource:0}: Error finding container 60174eaa2bd6d81588519bfe2a3f9e112d23b08816c4798bd1e07e61ad9b112d: Status 404 returned error can't find the container with id 60174eaa2bd6d81588519bfe2a3f9e112d23b08816c4798bd1e07e61ad9b112d
Jul 28 10:24:02.688844 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.688295    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xkrf6"]
Jul 28 10:24:02.810039 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.809998    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92" event={"ID":"bb033949-9ce9-4b56-a7e9-7f43383d5af3","Type":"ContainerStarted","Data":"60174eaa2bd6d81588519bfe2a3f9e112d23b08816c4798bd1e07e61ad9b112d"}
Jul 28 10:24:02.811617 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.811535    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xkrf6" event={"ID":"95553483-b6f2-47e4-a9dd-02266be7a816","Type":"ContainerStarted","Data":"20c8327c716e95c8264b396238dc830f441aef68da12c51ccc3f4492e5705b7d"}
Jul 28 10:24:02.813744 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.813717    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9d49bfb5-n7bxk" event={"ID":"435525f2-d692-4697-b299-0a6484bbd281","Type":"ContainerStarted","Data":"906e6a6fd7ab75a2232151b8301c205e60b353d07d1e4132c166b1b950c76e65"}
Jul 28 10:24:02.813862 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.813767    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9d49bfb5-n7bxk" event={"ID":"435525f2-d692-4697-b299-0a6484bbd281","Type":"ContainerStarted","Data":"f9df2373a4eb28057979f59bbf152cfd011d3956a8f68bbf8b606087eebf8079"}
Jul 28 10:24:02.815613 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.815586    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz" event={"ID":"daebe161-b184-41a5-9bb6-e41424e14559","Type":"ContainerStarted","Data":"c06453b652e0620636071be5acb4f23df4c216e73602012489caa6174daa236e"}
Jul 28 10:24:02.818139 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.818113    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" event={"ID":"048d224f-dacc-4fc0-ae66-9d750de55b6b","Type":"ContainerStarted","Data":"17935b01cd4dd566b69cf5f443c033f20db2fbb0594b333fe5a4733f14658835"}
Jul 28 10:24:02.818252 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.818154    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" event={"ID":"048d224f-dacc-4fc0-ae66-9d750de55b6b","Type":"ContainerStarted","Data":"19b47a38d8ba34665df12472163b89a7a3b5538ac477a4f0493983cb5ab3b600"}
Jul 28 10:24:02.818622 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.818587    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:24:02.820624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.820539    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m" event={"ID":"c17c2d08-5d58-4f09-be1c-07b53349c486","Type":"ContainerStarted","Data":"09a7ecbbfb359f275d9604ad19bcc44bafe9e222ad7e6acf1d0122274bad599a"}
Jul 28 10:24:02.835505 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.835319    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9d49bfb5-n7bxk" podStartSLOduration=38.835304553 podStartE2EDuration="38.835304553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:02.833412297 +0000 UTC m=+68.111349470" watchObservedRunningTime="2026-07-28 10:24:02.835304553 +0000 UTC m=+68.113241727"
Jul 28 10:24:02.856466 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:02.855527    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" podStartSLOduration=66.855490809 podStartE2EDuration="1m6.855490809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:02.854521678 +0000 UTC m=+68.132458851" watchObservedRunningTime="2026-07-28 10:24:02.855490809 +0000 UTC m=+68.133427984"
Jul 28 10:24:03.265453 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.265061    2544 scope.go:117] "RemoveContainer" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"
Jul 28 10:24:03.331476 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.330263    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:03.333494 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.333276    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:03.832350 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.832259    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9h54r" event={"ID":"f5dc3097-a4c6-4167-b237-9533efadbc76","Type":"ContainerStarted","Data":"359e1eefa7f3fc8b06d3894833cd1906dac75da34b3c5ad200fd913131515b2d"}
Jul 28 10:24:03.832350 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.832328    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9h54r" event={"ID":"f5dc3097-a4c6-4167-b237-9533efadbc76","Type":"ContainerStarted","Data":"882729adae24a5a061b3d1d947d2617abd7906df70a370523b4f15085c543290"}
Jul 28 10:24:03.838637 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.838609    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:24:03.840016 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.839977    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/1.log"
Jul 28 10:24:03.840105 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.840014    2544 generic.go:356] "Generic (PLEG): container finished" podID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37" containerID="aa18ecb7c95435d44b220882369ca3adb2f154b2a9a59826f27e2f69846a810b" exitCode=255
Jul 28 10:24:03.840924 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.840903    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" event={"ID":"b5ddfb23-d3c5-412f-a084-79ece8f6ac37","Type":"ContainerDied","Data":"aa18ecb7c95435d44b220882369ca3adb2f154b2a9a59826f27e2f69846a810b"}
Jul 28 10:24:03.841008 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.840954    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:03.841008 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.840982    2544 scope.go:117] "RemoveContainer" containerID="268eb3cc4cd254885e06a3c0f113cfb8ac368ed3803cf775df9286d39263fb1f"
Jul 28 10:24:03.841908 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.841892    2544 scope.go:117] "RemoveContainer" containerID="aa18ecb7c95435d44b220882369ca3adb2f154b2a9a59826f27e2f69846a810b"
Jul 28 10:24:03.842077 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:24:03.842052    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-bmpbj_openshift-console-operator(b5ddfb23-d3c5-412f-a084-79ece8f6ac37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podUID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37"
Jul 28 10:24:03.843944 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.843924    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9d49bfb5-n7bxk"
Jul 28 10:24:03.871143 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:03.870582    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9h54r" podStartSLOduration=67.197667143 podStartE2EDuration="1m8.870548641s" totalImagesPullingTime="1.672881498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:03.850800216 +0000 UTC m=+69.128737391" watchObservedRunningTime="2026-07-28 10:24:03.870548641 +0000 UTC m=+69.148485816"
Jul 28 10:24:04.847356 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:04.847323    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:24:10.506987 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:10.506939    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:24:10.507323 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:10.507001    2544 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:24:10.507369 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:10.507340    2544 scope.go:117] "RemoveContainer" containerID="aa18ecb7c95435d44b220882369ca3adb2f154b2a9a59826f27e2f69846a810b"
Jul 28 10:24:10.507525 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:24:10.507500    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-bmpbj_openshift-console-operator(b5ddfb23-d3c5-412f-a084-79ece8f6ac37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podUID="b5ddfb23-d3c5-412f-a084-79ece8f6ac37"
Jul 28 10:24:11.465236 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.465202    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"]
Jul 28 10:24:11.481723 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.481681    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:11.481834 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.481751    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"]
Jul 28 10:24:11.485453 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.485185    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qkcn4\""
Jul 28 10:24:11.485453 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.485194    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 10:24:11.601631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.601598    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/09292749-a301-4d15-ab37-48683a5f1949-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9ws4c\" (UID: \"09292749-a301-4d15-ab37-48683a5f1949\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:11.702674 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.702640    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/09292749-a301-4d15-ab37-48683a5f1949-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9ws4c\" (UID: \"09292749-a301-4d15-ab37-48683a5f1949\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:11.705133 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.705115    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/09292749-a301-4d15-ab37-48683a5f1949-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9ws4c\" (UID: \"09292749-a301-4d15-ab37-48683a5f1949\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:11.796218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.796184    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:11.880814 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.879266    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xkrf6" event={"ID":"95553483-b6f2-47e4-a9dd-02266be7a816","Type":"ContainerStarted","Data":"9a469f7b45421c50b90f9d42d1f9d45db6c472fdc2f8475c499f960403dce1ed"}
Jul 28 10:24:11.880814 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.879320    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xkrf6" event={"ID":"95553483-b6f2-47e4-a9dd-02266be7a816","Type":"ContainerStarted","Data":"c9e34be2aaa644e12cbb8d425bc04d43b75ef5b7f8e9b039ab0268cbfc0a4d19"}
Jul 28 10:24:11.881052 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.880879    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:24:11.882420 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.882300    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-67ksr" event={"ID":"0cefb3d3-e521-4c48-a063-77eb91f8ef91","Type":"ContainerStarted","Data":"0a4b70f15abf4b6b7624a4072ba99f9f4352c818ccae920fe391a04a5446b352"}
Jul 28 10:24:11.885575 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.885482    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nsnq9" event={"ID":"321c2faa-5209-4b2f-ae32-4e3bb51f1b38","Type":"ContainerStarted","Data":"b137dfaf6e9571cc8f0b112d7ff705f39ea71d9877ca077289ea78eac7855b51"}
Jul 28 10:24:11.887793 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.887773    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz" event={"ID":"daebe161-b184-41a5-9bb6-e41424e14559","Type":"ContainerStarted","Data":"9424d53c284273cd13792e1e0ad1883156501fc32c64458d98101f665fa9a6bf"}
Jul 28 10:24:11.890389 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.890368    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m" event={"ID":"c17c2d08-5d58-4f09-be1c-07b53349c486","Type":"ContainerStarted","Data":"6f51e704587b490cc6f125a78601983e15670823bd3d86195b59c8de2b2be198"}
Jul 28 10:24:11.890503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.890414    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m" event={"ID":"c17c2d08-5d58-4f09-be1c-07b53349c486","Type":"ContainerStarted","Data":"fb38520d8f8213f644033fb3fd1a1f4c0cf4bbb00260ce0feb1f942f9b44b5b2"}
Jul 28 10:24:11.892178 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.892159    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92" event={"ID":"bb033949-9ce9-4b56-a7e9-7f43383d5af3","Type":"ContainerStarted","Data":"1abb45087e2493b1836fc18e4c072f22fa5a4c21f7d4eacdcdb3cb48d49cd3dc"}
Jul 28 10:24:11.903835 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.903671    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xkrf6" podStartSLOduration=33.732213531 podStartE2EDuration="41.902851576s" totalImagesPullingTime="8.170638045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:11.899967163 +0000 UTC m=+77.177904338" watchObservedRunningTime="2026-07-28 10:24:11.902851576 +0000 UTC m=+77.180788750"
Jul 28 10:24:11.917652 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.917608    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bjq8m" podStartSLOduration=39.622849847 podStartE2EDuration="47.917591757s" totalImagesPullingTime="8.29474191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:11.916746529 +0000 UTC m=+77.194683704" watchObservedRunningTime="2026-07-28 10:24:11.917591757 +0000 UTC m=+77.195528931"
Jul 28 10:24:11.932156 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.932102    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dd6jz" podStartSLOduration=39.584141614 podStartE2EDuration="47.932058386s" totalImagesPullingTime="8.347916772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:11.931431855 +0000 UTC m=+77.209369030" watchObservedRunningTime="2026-07-28 10:24:11.932058386 +0000 UTC m=+77.209995560"
Jul 28 10:24:11.946496 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.946399    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-67ksr" podStartSLOduration=66.541602154 podStartE2EDuration="1m15.946379937s" totalImagesPullingTime="9.404777783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:11.945376951 +0000 UTC m=+77.223314125" watchObservedRunningTime="2026-07-28 10:24:11.946379937 +0000 UTC m=+77.224317111"
Jul 28 10:24:11.959374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.959331    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pnm92" podStartSLOduration=37.699151395 podStartE2EDuration="45.959302644s" totalImagesPullingTime="8.260151249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:11.958729277 +0000 UTC m=+77.236666454" watchObservedRunningTime="2026-07-28 10:24:11.959302644 +0000 UTC m=+77.237239818"
Jul 28 10:24:11.963702 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:11.963678    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"]
Jul 28 10:24:11.967138 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:11.967113    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09292749_a301_4d15_ab37_48683a5f1949.slice/crio-d7bed8ee6a31114afd61ab3d4b6a6c68aab1b901b4ce62ea834fa499b2a45828 WatchSource:0}: Error finding container d7bed8ee6a31114afd61ab3d4b6a6c68aab1b901b4ce62ea834fa499b2a45828: Status 404 returned error can't find the container with id d7bed8ee6a31114afd61ab3d4b6a6c68aab1b901b4ce62ea834fa499b2a45828
Jul 28 10:24:12.902706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:12.902671    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c" event={"ID":"09292749-a301-4d15-ab37-48683a5f1949","Type":"ContainerStarted","Data":"d7bed8ee6a31114afd61ab3d4b6a6c68aab1b901b4ce62ea834fa499b2a45828"}
Jul 28 10:24:15.915761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:15.915726    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c" event={"ID":"09292749-a301-4d15-ab37-48683a5f1949","Type":"ContainerStarted","Data":"d56df07c0dd69372c1848f081daaafeaa3c024d8c14feb8c9f45345319cd94e9"}
Jul 28 10:24:15.916214 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:15.915811    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:15.920528 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:15.920506    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c"
Jul 28 10:24:15.932969 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:15.932926    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9ws4c" podStartSLOduration=1.179275893 podStartE2EDuration="4.932910422s" totalImagesPullingTime="3.753634529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:15.931532397 +0000 UTC m=+81.209469572" watchObservedRunningTime="2026-07-28 10:24:15.932910422 +0000 UTC m=+81.210847633"
Jul 28 10:24:16.922177 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:16.922051    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nsnq9" event={"ID":"321c2faa-5209-4b2f-ae32-4e3bb51f1b38","Type":"ContainerStarted","Data":"1ee044182e8541e535e3d09ab22445e816d65e68ec0303774f9065fbf4387e5e"}
Jul 28 10:24:20.939613 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:20.939553    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nsnq9" event={"ID":"321c2faa-5209-4b2f-ae32-4e3bb51f1b38","Type":"ContainerStarted","Data":"09ef6465357fb703e6be47912415754426145cf4d067d66783c8d87fa63af1ea"}
Jul 28 10:24:20.959201 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:20.959156    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nsnq9" podStartSLOduration=17.601721524 podStartE2EDuration="36.959142004s" totalImagesPullingTime="19.35742048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:23:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:20.956927798 +0000 UTC m=+86.234864973" watchObservedRunningTime="2026-07-28 10:24:20.959142004 +0000 UTC m=+86.237079178"
Jul 28 10:24:21.906319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:21.906284    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xkrf6"
Jul 28 10:24:22.197641 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:22.197527    2544 patch_prober.go:28] interesting pod/image-registry-7577b96c84-t5gcn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 10:24:22.198045 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:22.197626    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 10:24:24.853885 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:24.853858    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:24:25.267047 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.266979    2544 scope.go:117] "RemoveContainer" containerID="aa18ecb7c95435d44b220882369ca3adb2f154b2a9a59826f27e2f69846a810b"
Jul 28 10:24:25.965334 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.965308    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:24:25.965775 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.965379    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" event={"ID":"b5ddfb23-d3c5-412f-a084-79ece8f6ac37","Type":"ContainerStarted","Data":"896cac305f1de8650626e17664dd3dcdf62f8924cb5195a9c8b5bd18acd7cbcc"}
Jul 28 10:24:25.965775 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.965668    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:24:25.970459 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.970441    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj"
Jul 28 10:24:25.984082 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:25.984026    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-bmpbj" podStartSLOduration=50.079985039 podStartE2EDuration="1m1.984008147s" totalImagesPullingTime="11.904023108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:25.982654081 +0000 UTC m=+91.260591256" watchObservedRunningTime="2026-07-28 10:24:25.984008147 +0000 UTC m=+91.261945322"
Jul 28 10:24:27.903633 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.903602    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c25pz"]
Jul 28 10:24:27.907930 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.907909    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.911492 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.911318    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 10:24:27.911894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.911874    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 10:24:27.913470 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.913301    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 10:24:27.913470 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.913355    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 10:24:27.915283 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.915239    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qpdwx\""
Jul 28 10:24:27.937017 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.936997    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wpr2\" (UniqueName: \"kubernetes.io/projected/c77abb78-4a3c-4dd4-afbc-fc34770abee5-kube-api-access-4wpr2\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937198 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937181    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-metrics-client-ca\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937352 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937330    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937489 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937376    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-textfile\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937489 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937406    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937489 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937448    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937489 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937478    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-root\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937807 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937501    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-wtmp\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:27.937807 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:27.937531    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-sys\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038552    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-sys\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038643    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wpr2\" (UniqueName: \"kubernetes.io/projected/c77abb78-4a3c-4dd4-afbc-fc34770abee5-kube-api-access-4wpr2\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038672    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-metrics-client-ca\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038710    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038779 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038750    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-textfile\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038991 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038782    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.038991 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.038825    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039339 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039314    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-metrics-client-ca\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039438 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039333    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-sys\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039438 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039374    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-root\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039438 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039402    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-wtmp\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039604 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039581    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-wtmp\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039665 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039612    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039665 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.039626    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c77abb78-4a3c-4dd4-afbc-fc34770abee5-root\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.039768 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:24:28.039706    2544 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 10:24:28.039898 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:24:28.039883    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls podName:c77abb78-4a3c-4dd4-afbc-fc34770abee5 nodeName:}" failed. No retries permitted until 2026-07-28 10:24:28.539852429 +0000 UTC m=+93.817789581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls") pod "node-exporter-c25pz" (UID: "c77abb78-4a3c-4dd4-afbc-fc34770abee5") : secret "node-exporter-tls" not found
Jul 28 10:24:28.040044 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.040013    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-textfile\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.042133 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.042111    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.049529 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.049507    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wpr2\" (UniqueName: \"kubernetes.io/projected/c77abb78-4a3c-4dd4-afbc-fc34770abee5-kube-api-access-4wpr2\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.543386 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.543354    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.545809 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.545784    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c77abb78-4a3c-4dd4-afbc-fc34770abee5-node-exporter-tls\") pod \"node-exporter-c25pz\" (UID: \"c77abb78-4a3c-4dd4-afbc-fc34770abee5\") " pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.829337 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.829252    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c25pz"
Jul 28 10:24:28.840286 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:24:28.840250    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77abb78_4a3c_4dd4_afbc_fc34770abee5.slice/crio-616540a22f4be2a7e1455f128a9aa9be33c0eb0c6bea58b26eca3eb475926b2c WatchSource:0}: Error finding container 616540a22f4be2a7e1455f128a9aa9be33c0eb0c6bea58b26eca3eb475926b2c: Status 404 returned error can't find the container with id 616540a22f4be2a7e1455f128a9aa9be33c0eb0c6bea58b26eca3eb475926b2c
Jul 28 10:24:28.979076 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:28.979043    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25pz" event={"ID":"c77abb78-4a3c-4dd4-afbc-fc34770abee5","Type":"ContainerStarted","Data":"616540a22f4be2a7e1455f128a9aa9be33c0eb0c6bea58b26eca3eb475926b2c"}
Jul 28 10:24:29.984141 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:29.984112    2544 generic.go:356] "Generic (PLEG): container finished" podID="c77abb78-4a3c-4dd4-afbc-fc34770abee5" containerID="da3766997e395b6c7bac2a6815e7afac9f0c2983b90052079e4a3eb04bfb82f6" exitCode=0
Jul 28 10:24:29.984461 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:29.984151    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25pz" event={"ID":"c77abb78-4a3c-4dd4-afbc-fc34770abee5","Type":"ContainerDied","Data":"da3766997e395b6c7bac2a6815e7afac9f0c2983b90052079e4a3eb04bfb82f6"}
Jul 28 10:24:30.990610 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:30.990550    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25pz" event={"ID":"c77abb78-4a3c-4dd4-afbc-fc34770abee5","Type":"ContainerStarted","Data":"75a0139ab097d4f417425134963f150f2b2f03b7c16d29edb7307928720be921"}
Jul 28 10:24:30.991005 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:30.990640    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25pz" event={"ID":"c77abb78-4a3c-4dd4-afbc-fc34770abee5","Type":"ContainerStarted","Data":"c983feff5f37f2d49af02bf15139b7ea50414da22a16a9b30e591571edbadcb7"}
Jul 28 10:24:31.020263 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:31.020206    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c25pz" podStartSLOduration=3.025822492 podStartE2EDuration="4.02018607s" totalImagesPullingTime="921.161337ms" totalInitContainerRuntime="73.202241ms" isStatefulPod=true podCreationTimestamp="2026-07-28 10:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:24:31.018624697 +0000 UTC m=+96.296561874" watchObservedRunningTime="2026-07-28 10:24:31.02018607 +0000 UTC m=+96.298123247"
Jul 28 10:24:32.822970 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:32.822937    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mzjkp"
Jul 28 10:24:37.929319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:37.929283    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:24:49.058924 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:49.058892    2544 generic.go:356] "Generic (PLEG): container finished" podID="662b8125-a009-4bad-ba14-574df1776a3e" containerID="4fb79a0addb68d617053167ab3b6e6c1f0dfb36c1816ab5a06e3fb77dbee421a" exitCode=0
Jul 28 10:24:49.059397 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:49.058969    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl" event={"ID":"662b8125-a009-4bad-ba14-574df1776a3e","Type":"ContainerDied","Data":"4fb79a0addb68d617053167ab3b6e6c1f0dfb36c1816ab5a06e3fb77dbee421a"}
Jul 28 10:24:49.059480 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:49.059350    2544 scope.go:117] "RemoveContainer" containerID="4fb79a0addb68d617053167ab3b6e6c1f0dfb36c1816ab5a06e3fb77dbee421a"
Jul 28 10:24:50.064154 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:50.064117    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl" event={"ID":"662b8125-a009-4bad-ba14-574df1776a3e","Type":"ContainerStarted","Data":"08a4d9717cdd9670e2208afdabbf0aee1d64052c1f752906876b4329d68517ba"}
Jul 28 10:24:54.081659 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:54.081628    2544 generic.go:356] "Generic (PLEG): container finished" podID="3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122" containerID="0be2f5ae97feb1af78735462f3f4407952b70a27d139f27d1bf54b44b98e60f0" exitCode=0
Jul 28 10:24:54.082069 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:54.081711    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-gg27m" event={"ID":"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122","Type":"ContainerDied","Data":"0be2f5ae97feb1af78735462f3f4407952b70a27d139f27d1bf54b44b98e60f0"}
Jul 28 10:24:54.082069 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:54.082011    2544 scope.go:117] "RemoveContainer" containerID="0be2f5ae97feb1af78735462f3f4407952b70a27d139f27d1bf54b44b98e60f0"
Jul 28 10:24:55.087364 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:24:55.087330    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-gg27m" event={"ID":"3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122","Type":"ContainerStarted","Data":"e4318dd798d0ff955075cf4f4040f7f168e12df4642d9ddb64d3cfd31b489998"}
Jul 28 10:25:02.949668 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:02.949603    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerName="registry" containerID="cri-o://17935b01cd4dd566b69cf5f443c033f20db2fbb0594b333fe5a4733f14658835" gracePeriod=30
Jul 28 10:25:03.119677 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.119644    2544 generic.go:356] "Generic (PLEG): container finished" podID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerID="17935b01cd4dd566b69cf5f443c033f20db2fbb0594b333fe5a4733f14658835" exitCode=0
Jul 28 10:25:03.119832 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.119711    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" event={"ID":"048d224f-dacc-4fc0-ae66-9d750de55b6b","Type":"ContainerDied","Data":"17935b01cd4dd566b69cf5f443c033f20db2fbb0594b333fe5a4733f14658835"}
Jul 28 10:25:03.196510 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.196489    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:25:03.313131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313058    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313088    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313131 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313118    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313384 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313165    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313384 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313204    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313384 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313278    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313384 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313311    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313384 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313341    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp4mc\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc\") pod \"048d224f-dacc-4fc0-ae66-9d750de55b6b\" (UID: \"048d224f-dacc-4fc0-ae66-9d750de55b6b\") "
Jul 28 10:25:03.313742 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.313710    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 10:25:03.314148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.314054    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 10:25:03.315844 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.315794    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:25:03.315961 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.315894    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:25:03.316068 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.316048    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:25:03.316118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.316057    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:25:03.316355 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.316337    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc" (OuterVolumeSpecName: "kube-api-access-kp4mc") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "kube-api-access-kp4mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:25:03.321711 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.321687    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "048d224f-dacc-4fc0-ae66-9d750de55b6b" (UID: "048d224f-dacc-4fc0-ae66-9d750de55b6b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:25:03.414121 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414093    2544 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-installation-pull-secrets\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414121 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414115    2544 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-certificates\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414121 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414125    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp4mc\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-kube-api-access-kp4mc\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414136    2544 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/048d224f-dacc-4fc0-ae66-9d750de55b6b-ca-trust-extracted\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414144    2544 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-bound-sa-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414154    2544 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/048d224f-dacc-4fc0-ae66-9d750de55b6b-trusted-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414161    2544 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/048d224f-dacc-4fc0-ae66-9d750de55b6b-registry-tls\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:03.414375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:03.414169    2544 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/048d224f-dacc-4fc0-ae66-9d750de55b6b-image-registry-private-configuration\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:25:04.124797 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:04.124769    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn"
Jul 28 10:25:04.125214 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:04.124766    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7577b96c84-t5gcn" event={"ID":"048d224f-dacc-4fc0-ae66-9d750de55b6b","Type":"ContainerDied","Data":"19b47a38d8ba34665df12472163b89a7a3b5538ac477a4f0493983cb5ab3b600"}
Jul 28 10:25:04.125214 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:04.124927    2544 scope.go:117] "RemoveContainer" containerID="17935b01cd4dd566b69cf5f443c033f20db2fbb0594b333fe5a4733f14658835"
Jul 28 10:25:04.149090 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:04.149068    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:25:04.151692 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:04.151671    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7577b96c84-t5gcn"]
Jul 28 10:25:05.268887 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:05.268857    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" path="/var/lib/kubelet/pods/048d224f-dacc-4fc0-ae66-9d750de55b6b/volumes"
Jul 28 10:25:14.170484 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:14.170455    2544 generic.go:356] "Generic (PLEG): container finished" podID="79027776-4d57-4fac-a422-0a2cce20c33f" containerID="82cd4bc7b4fe2fe9e52e68cba2dcbefba86441802ab6ee62d7360138ee263b52" exitCode=0
Jul 28 10:25:14.170812 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:14.170501    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl" event={"ID":"79027776-4d57-4fac-a422-0a2cce20c33f","Type":"ContainerDied","Data":"82cd4bc7b4fe2fe9e52e68cba2dcbefba86441802ab6ee62d7360138ee263b52"}
Jul 28 10:25:14.170812 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:14.170804    2544 scope.go:117] "RemoveContainer" containerID="82cd4bc7b4fe2fe9e52e68cba2dcbefba86441802ab6ee62d7360138ee263b52"
Jul 28 10:25:15.177292 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:15.177258    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-8gnbl" event={"ID":"79027776-4d57-4fac-a422-0a2cce20c33f","Type":"ContainerStarted","Data":"af6a2273c79efdf7eb98b4e8515b1ef0b44372ba13cb227311992b851415fc3c"}
Jul 28 10:25:20.596207 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:20.596166    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" podUID="9977dae4-55ad-4243-b0af-1fa676491f00" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 10:25:30.596052 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:30.596013    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" podUID="9977dae4-55ad-4243-b0af-1fa676491f00" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 10:25:40.596284 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:40.596241    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" podUID="9977dae4-55ad-4243-b0af-1fa676491f00" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 10:25:40.596669 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:40.596320    2544 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv"
Jul 28 10:25:40.596831 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:40.596803    2544 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"156e2085f3a3ff50a2d0ffacfb4008e04a3e4209dfaa5e9375cf0866ba46ba47"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 28 10:25:40.596867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:40.596852    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" podUID="9977dae4-55ad-4243-b0af-1fa676491f00" containerName="service-proxy" containerID="cri-o://156e2085f3a3ff50a2d0ffacfb4008e04a3e4209dfaa5e9375cf0866ba46ba47" gracePeriod=30
Jul 28 10:25:41.280379 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:41.280340    2544 generic.go:356] "Generic (PLEG): container finished" podID="9977dae4-55ad-4243-b0af-1fa676491f00" containerID="156e2085f3a3ff50a2d0ffacfb4008e04a3e4209dfaa5e9375cf0866ba46ba47" exitCode=2
Jul 28 10:25:41.280605 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:41.280390    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerDied","Data":"156e2085f3a3ff50a2d0ffacfb4008e04a3e4209dfaa5e9375cf0866ba46ba47"}
Jul 28 10:25:41.280605 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:25:41.280428    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cd48d4d8-27lpv" event={"ID":"9977dae4-55ad-4243-b0af-1fa676491f00","Type":"ContainerStarted","Data":"3398cf87dd7a5d11186096a1c0720607a47be2449521a900d45d6ac9d060d3ba"}
Jul 28 10:27:55.137280 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:27:55.137253    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:27:55.140730 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:27:55.140706    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:27:55.154489 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:27:55.154469    2544 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 10:32:55.169803 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:32:55.169771    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:32:55.170789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:32:55.170767    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:35:07.667281 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.667204    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"]
Jul 28 10:35:07.667688 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.667524    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerName="registry"
Jul 28 10:35:07.667688 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.667534    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerName="registry"
Jul 28 10:35:07.667688 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.667625    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="048d224f-dacc-4fc0-ae66-9d750de55b6b" containerName="registry"
Jul 28 10:35:07.670622 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.670607    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.674901 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.674883    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 10:35:07.675415 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.675400    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 10:35:07.675680 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.675636    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 10:35:07.676033 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.676019    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 10:35:07.676460 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.676446    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k7g2n\""
Jul 28 10:35:07.693829 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.693804    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"]
Jul 28 10:35:07.776135 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.776101    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.776135 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.776133    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-manifests\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.776357 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.776150    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.776357 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.776176    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-tmp\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.776357 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.776248    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n647p\" (UniqueName: \"kubernetes.io/projected/953e374c-964b-40a6-8ec5-3dd076748f39-kube-api-access-n647p\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877432 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877395    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n647p\" (UniqueName: \"kubernetes.io/projected/953e374c-964b-40a6-8ec5-3dd076748f39-kube-api-access-n647p\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877477    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877497    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-manifests\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877514    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877551    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-tmp\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.877899 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.877879    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-manifests\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.879976 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.879949    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953e374c-964b-40a6-8ec5-3dd076748f39-tmp\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.880284 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.880265    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.880368 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.880314    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/953e374c-964b-40a6-8ec5-3dd076748f39-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.885759 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.885741    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n647p\" (UniqueName: \"kubernetes.io/projected/953e374c-964b-40a6-8ec5-3dd076748f39-kube-api-access-n647p\") pod \"opendatahub-operator-controller-manager-559f66f7dc-hb2m2\" (UID: \"953e374c-964b-40a6-8ec5-3dd076748f39\") " pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:07.980947 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:07.980870    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:08.123010 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:08.122988    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"]
Jul 28 10:35:08.125723 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:08.125695    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953e374c_964b_40a6_8ec5_3dd076748f39.slice/crio-726dac31d96eedfa1d9d2fb9295c5613c796a5d760ebe698c56392f90ebe1ffd WatchSource:0}: Error finding container 726dac31d96eedfa1d9d2fb9295c5613c796a5d760ebe698c56392f90ebe1ffd: Status 404 returned error can't find the container with id 726dac31d96eedfa1d9d2fb9295c5613c796a5d760ebe698c56392f90ebe1ffd
Jul 28 10:35:08.127511 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:08.127497    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 10:35:08.436429 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:08.436402    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2" event={"ID":"953e374c-964b-40a6-8ec5-3dd076748f39","Type":"ContainerStarted","Data":"726dac31d96eedfa1d9d2fb9295c5613c796a5d760ebe698c56392f90ebe1ffd"}
Jul 28 10:35:11.453897 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:11.453860    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2" event={"ID":"953e374c-964b-40a6-8ec5-3dd076748f39","Type":"ContainerStarted","Data":"06db4e7282e4ab2b663067b68fbcb1c0084d8d7e5ef7baefc9ef4304f3456b2e"}
Jul 28 10:35:12.460278 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:12.460194    2544 generic.go:356] "Generic (PLEG): container finished" podID="953e374c-964b-40a6-8ec5-3dd076748f39" containerID="06db4e7282e4ab2b663067b68fbcb1c0084d8d7e5ef7baefc9ef4304f3456b2e" exitCode=0
Jul 28 10:35:12.460682 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:12.460277    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2" event={"ID":"953e374c-964b-40a6-8ec5-3dd076748f39","Type":"ContainerDied","Data":"06db4e7282e4ab2b663067b68fbcb1c0084d8d7e5ef7baefc9ef4304f3456b2e"}
Jul 28 10:35:13.466121 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:13.466085    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2" event={"ID":"953e374c-964b-40a6-8ec5-3dd076748f39","Type":"ContainerStarted","Data":"a6fb11e58f867921baad98f1261959d9398c4d36a32757312d4692d175ccc62f"}
Jul 28 10:35:13.466611 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:13.466186    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:13.491504 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:13.491450    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2" podStartSLOduration=2.177469774 podStartE2EDuration="6.491432872s" totalImagesPullingTime="3.240254329s" totalInitContainerRuntime="1.073708769s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:13.488989317 +0000 UTC m=+738.766926530" watchObservedRunningTime="2026-07-28 10:35:13.491432872 +0000 UTC m=+738.769370047"
Jul 28 10:35:24.473221 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:24.473190    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-559f66f7dc-hb2m2"
Jul 28 10:35:26.751664 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.751626    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfj2q"]
Jul 28 10:35:26.756927 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.756905    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.759953 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.759924    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 10:35:26.760071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.760041    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-p8bd4\""
Jul 28 10:35:26.774731 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.774703    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfj2q"]
Jul 28 10:35:26.838145 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.838108    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9tms\" (UniqueName: \"kubernetes.io/projected/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-kube-api-access-j9tms\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.838323 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.838182    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-cert\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.939118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.939085    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-cert\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.939306 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.939205    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9tms\" (UniqueName: \"kubernetes.io/projected/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-kube-api-access-j9tms\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.941638 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.941612    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-cert\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:26.950035 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:26.950016    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9tms\" (UniqueName: \"kubernetes.io/projected/9d0cc820-c243-41de-8bf8-d8b17b3c36f8-kube-api-access-j9tms\") pod \"odh-model-controller-7998c67dc4-zfj2q\" (UID: \"9d0cc820-c243-41de-8bf8-d8b17b3c36f8\") " pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:27.067255 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.067179    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:27.205950 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.205918    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zfj2q"]
Jul 28 10:35:27.208830 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:27.208792    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0cc820_c243_41de_8bf8_d8b17b3c36f8.slice/crio-341d7ee17e2a81a3f2036fca665afd610720e6ab707b550b02565ad0f6dbbf54 WatchSource:0}: Error finding container 341d7ee17e2a81a3f2036fca665afd610720e6ab707b550b02565ad0f6dbbf54: Status 404 returned error can't find the container with id 341d7ee17e2a81a3f2036fca665afd610720e6ab707b550b02565ad0f6dbbf54
Jul 28 10:35:27.381259 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.381219    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5mh2r"]
Jul 28 10:35:27.383886 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.383871    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:27.386362 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.386342    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cxqcl\""
Jul 28 10:35:27.386469 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.386341    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 10:35:27.396412 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.396386    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5mh2r"]
Jul 28 10:35:27.528255 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.528222    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q" event={"ID":"9d0cc820-c243-41de-8bf8-d8b17b3c36f8","Type":"ContainerStarted","Data":"341d7ee17e2a81a3f2036fca665afd610720e6ab707b550b02565ad0f6dbbf54"}
Jul 28 10:35:27.543632 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.543599    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz9rt\" (UniqueName: \"kubernetes.io/projected/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-kube-api-access-qz9rt\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:27.543759 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.543660    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:27.644995 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.644904    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz9rt\" (UniqueName: \"kubernetes.io/projected/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-kube-api-access-qz9rt\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:27.644995 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.644985    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:27.645177 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:35:27.645127    2544 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 10:35:27.645223 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:35:27.645213    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs podName:a8c05a0a-bbec-46f7-bb87-aaa8685a5b77 nodeName:}" failed. No retries permitted until 2026-07-28 10:35:28.145197376 +0000 UTC m=+753.423134528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs") pod "model-serving-api-54d89bd5fc-5mh2r" (UID: "a8c05a0a-bbec-46f7-bb87-aaa8685a5b77") : secret "model-serving-api-tls" not found
Jul 28 10:35:27.653966 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:27.653939    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz9rt\" (UniqueName: \"kubernetes.io/projected/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-kube-api-access-qz9rt\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:28.150342 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:28.150295    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:28.153452 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:28.153423    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c05a0a-bbec-46f7-bb87-aaa8685a5b77-tls-certs\") pod \"model-serving-api-54d89bd5fc-5mh2r\" (UID: \"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77\") " pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:28.294574 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:28.294524    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:28.461509 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:28.461387    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5mh2r"]
Jul 28 10:35:28.468219 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:28.466400    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c05a0a_bbec_46f7_bb87_aaa8685a5b77.slice/crio-5535e8d215f560cbe360694944c25738edc82bfef7fe6015a7ea274191482119 WatchSource:0}: Error finding container 5535e8d215f560cbe360694944c25738edc82bfef7fe6015a7ea274191482119: Status 404 returned error can't find the container with id 5535e8d215f560cbe360694944c25738edc82bfef7fe6015a7ea274191482119
Jul 28 10:35:28.535409 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:28.535368    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r" event={"ID":"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77","Type":"ContainerStarted","Data":"5535e8d215f560cbe360694944c25738edc82bfef7fe6015a7ea274191482119"}
Jul 28 10:35:30.546984 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:30.546948    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q" event={"ID":"9d0cc820-c243-41de-8bf8-d8b17b3c36f8","Type":"ContainerStarted","Data":"2e79bab7009d93b06cdf0897ee6f01878836fc4f8c40a3aaf13b386f999b82e3"}
Jul 28 10:35:30.547447 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:30.547056    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:30.563166 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:30.562970    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q" podStartSLOduration=1.839617016 podStartE2EDuration="4.562957996s" totalImagesPullingTime="2.72334098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:30.562769801 +0000 UTC m=+755.840706975" watchObservedRunningTime="2026-07-28 10:35:30.562957996 +0000 UTC m=+755.840895169"
Jul 28 10:35:31.552833 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:31.552796    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r" event={"ID":"a8c05a0a-bbec-46f7-bb87-aaa8685a5b77","Type":"ContainerStarted","Data":"07290f2ba1ce0c050edff4f135e24a076646e001db4c2a7c9fe58b338d0fe197"}
Jul 28 10:35:31.553253 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:31.553046    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:31.577479 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:31.577438    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r" podStartSLOduration=2.009244476 podStartE2EDuration="4.577425299s" totalImagesPullingTime="2.568180823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:31.576635963 +0000 UTC m=+756.854573136" watchObservedRunningTime="2026-07-28 10:35:31.577425299 +0000 UTC m=+756.855362475"
Jul 28 10:35:36.564299 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.564268    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"]
Jul 28 10:35:36.567810 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.567789    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.570268 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.570244    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jnkxr\""
Jul 28 10:35:36.570500 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.570481    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 10:35:36.582228 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.582206    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"]
Jul 28 10:35:36.585468 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.585446    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.587425 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.587405    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"]
Jul 28 10:35:36.588105 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.588086    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 10:35:36.588424 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.588406    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-26hgs\""
Jul 28 10:35:36.596307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.596288    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"]
Jul 28 10:35:36.624689 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.624664    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqdtf\" (UniqueName: \"kubernetes.io/projected/f8cda8cc-33ad-41e4-bd33-3a8f66243322-kube-api-access-rqdtf\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.624774 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.624716    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8cda8cc-33ad-41e4-bd33-3a8f66243322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.725187 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.725156    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqdtf\" (UniqueName: \"kubernetes.io/projected/f8cda8cc-33ad-41e4-bd33-3a8f66243322-kube-api-access-rqdtf\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.725187 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.725197    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp2z4\" (UniqueName: \"kubernetes.io/projected/f0f135df-5161-4f47-b431-8dca8b419610-kube-api-access-sp2z4\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.725391 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.725234    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0f135df-5161-4f47-b431-8dca8b419610-cert\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.725391 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.725324    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8cda8cc-33ad-41e4-bd33-3a8f66243322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.728039 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.728014    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8cda8cc-33ad-41e4-bd33-3a8f66243322-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.738722 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.738697    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqdtf\" (UniqueName: \"kubernetes.io/projected/f8cda8cc-33ad-41e4-bd33-3a8f66243322-kube-api-access-rqdtf\") pod \"llmisvc-controller-manager-777c6bd7d5-5k7t4\" (UID: \"f8cda8cc-33ad-41e4-bd33-3a8f66243322\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:36.826479 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.826382    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0f135df-5161-4f47-b431-8dca8b419610-cert\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.826479 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.826474    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp2z4\" (UniqueName: \"kubernetes.io/projected/f0f135df-5161-4f47-b431-8dca8b419610-kube-api-access-sp2z4\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.828993 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.828969    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0f135df-5161-4f47-b431-8dca8b419610-cert\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.854387 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.854356    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp2z4\" (UniqueName: \"kubernetes.io/projected/f0f135df-5161-4f47-b431-8dca8b419610-kube-api-access-sp2z4\") pod \"kserve-controller-manager-d6f4b65b8-9whhf\" (UID: \"f0f135df-5161-4f47-b431-8dca8b419610\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.879076 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.879050    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:36.895667 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:36.895647    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:37.096579 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:37.096465    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f135df_5161_4f47_b431_8dca8b419610.slice/crio-2da6503a81859eac93ea7fe257dc3f40314eef073947ea8b9686215b52d14c82 WatchSource:0}: Error finding container 2da6503a81859eac93ea7fe257dc3f40314eef073947ea8b9686215b52d14c82: Status 404 returned error can't find the container with id 2da6503a81859eac93ea7fe257dc3f40314eef073947ea8b9686215b52d14c82
Jul 28 10:35:37.098502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.098416    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"]
Jul 28 10:35:37.108145 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.108124    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"]
Jul 28 10:35:37.114403 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:37.114379    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8cda8cc_33ad_41e4_bd33_3a8f66243322.slice/crio-a081c2476704de3afb2a312ff6333514ee003dc6c3252ac936b865275441220d WatchSource:0}: Error finding container a081c2476704de3afb2a312ff6333514ee003dc6c3252ac936b865275441220d: Status 404 returned error can't find the container with id a081c2476704de3afb2a312ff6333514ee003dc6c3252ac936b865275441220d
Jul 28 10:35:37.263160 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.263131    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"]
Jul 28 10:35:37.267829 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.267806    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.270409 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.270386    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 10:35:37.270530 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.270431    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ntp8r\""
Jul 28 10:35:37.272502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.272473    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 10:35:37.284105 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.284082    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"]
Jul 28 10:35:37.331307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.331272    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca029121-a68d-4531-9152-23bf1a84c263-tls-certs\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.331307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.331305    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca029121-a68d-4531-9152-23bf1a84c263-tmp\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.331624 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.331326    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9577d\" (UniqueName: \"kubernetes.io/projected/ca029121-a68d-4531-9152-23bf1a84c263-kube-api-access-9577d\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.432387 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.432360    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca029121-a68d-4531-9152-23bf1a84c263-tls-certs\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.432538 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.432394    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca029121-a68d-4531-9152-23bf1a84c263-tmp\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.432538 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.432415    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9577d\" (UniqueName: \"kubernetes.io/projected/ca029121-a68d-4531-9152-23bf1a84c263-kube-api-access-9577d\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.434878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.434858    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca029121-a68d-4531-9152-23bf1a84c263-tmp\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.435090 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.435073    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca029121-a68d-4531-9152-23bf1a84c263-tls-certs\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.440445 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.440423    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9577d\" (UniqueName: \"kubernetes.io/projected/ca029121-a68d-4531-9152-23bf1a84c263-kube-api-access-9577d\") pod \"kube-auth-proxy-5d6b86846-rkl5w\" (UID: \"ca029121-a68d-4531-9152-23bf1a84c263\") " pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.579320 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.579294    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"
Jul 28 10:35:37.581315 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.581291    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4" event={"ID":"f8cda8cc-33ad-41e4-bd33-3a8f66243322","Type":"ContainerStarted","Data":"a081c2476704de3afb2a312ff6333514ee003dc6c3252ac936b865275441220d"}
Jul 28 10:35:37.582399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.582382    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf" event={"ID":"f0f135df-5161-4f47-b431-8dca8b419610","Type":"ContainerStarted","Data":"2da6503a81859eac93ea7fe257dc3f40314eef073947ea8b9686215b52d14c82"}
Jul 28 10:35:37.735204 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:37.735152    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w"]
Jul 28 10:35:37.737664 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:37.737629    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca029121_a68d_4531_9152_23bf1a84c263.slice/crio-9895f2fd16d8bf6e399b3a7bcfaab2277d967a3ed867906d3110b0fd8415a8bc WatchSource:0}: Error finding container 9895f2fd16d8bf6e399b3a7bcfaab2277d967a3ed867906d3110b0fd8415a8bc: Status 404 returned error can't find the container with id 9895f2fd16d8bf6e399b3a7bcfaab2277d967a3ed867906d3110b0fd8415a8bc
Jul 28 10:35:38.589709 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:38.589627    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w" event={"ID":"ca029121-a68d-4531-9152-23bf1a84c263","Type":"ContainerStarted","Data":"9895f2fd16d8bf6e399b3a7bcfaab2277d967a3ed867906d3110b0fd8415a8bc"}
Jul 28 10:35:41.555735 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.555699    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zfj2q"
Jul 28 10:35:41.607398 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.607360    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4" event={"ID":"f8cda8cc-33ad-41e4-bd33-3a8f66243322","Type":"ContainerStarted","Data":"ba6db9d8277e7d06c51b7e636547098c20666b8c11bf5e1ea40b04ccee71a233"}
Jul 28 10:35:41.607606 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.607461    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:35:41.609222 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.609192    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf" event={"ID":"f0f135df-5161-4f47-b431-8dca8b419610","Type":"ContainerStarted","Data":"8a36cbbe99ab66254db00629d17d8f81e51d90c89f3e3cd8de4d82d723fa52d2"}
Jul 28 10:35:41.609342 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.609320    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:35:41.610867 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.610847    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w" event={"ID":"ca029121-a68d-4531-9152-23bf1a84c263","Type":"ContainerStarted","Data":"5366e6658517a219d74fdec0ce04fb68c73918ee0a5e0e2c5e24955405ea514e"}
Jul 28 10:35:41.630662 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.630541    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4" podStartSLOduration=1.8610967999999999 podStartE2EDuration="5.630525785s" totalImagesPullingTime="3.769428985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:41.630237671 +0000 UTC m=+766.908174845" watchObservedRunningTime="2026-07-28 10:35:41.630525785 +0000 UTC m=+766.908462954"
Jul 28 10:35:41.651727 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.651665    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d6b86846-rkl5w" podStartSLOduration=1.499772071 podStartE2EDuration="4.651654118s" totalImagesPullingTime="3.151882047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:41.651002026 +0000 UTC m=+766.928939201" watchObservedRunningTime="2026-07-28 10:35:41.651654118 +0000 UTC m=+766.929591292"
Jul 28 10:35:41.672997 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:41.672938    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf" podStartSLOduration=1.8900500390000001 podStartE2EDuration="5.67291799s" totalImagesPullingTime="3.782867951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:41.671269176 +0000 UTC m=+766.949206351" watchObservedRunningTime="2026-07-28 10:35:41.67291799 +0000 UTC m=+766.950855165"
Jul 28 10:35:42.561916 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:42.561885    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5mh2r"
Jul 28 10:35:43.758408 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.758367    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"]
Jul 28 10:35:43.761996 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.761974    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.767740 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.767721    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cn25l\""
Jul 28 10:35:43.768459 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.768443    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 10:35:43.780474 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.780455    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"]
Jul 28 10:35:43.785699 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785672    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785811 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785714    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785811 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785789    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785937 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785823    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785937 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785861    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785937 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785884    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.785937 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785925    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.786143 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785947    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbrdt\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-kube-api-access-qbrdt\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.786143 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.785971    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887163 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887131    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887169    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887198    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887226    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887280    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887302    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbrdt\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-kube-api-access-qbrdt\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887621 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887361    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887621 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887400    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887621 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887433    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887774 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887658    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887774 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887678    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887774 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887761    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.887985 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.887966    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.888254 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.888232    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.889899 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.889883    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.890335 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.890314    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.905127 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.905104    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbrdt\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-kube-api-access-qbrdt\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:43.905620 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:43.905597    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035a1d4f-39a7-4f52-94c3-2ac37fdaed68-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-77785\" (UID: \"035a1d4f-39a7-4f52-94c3-2ac37fdaed68\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:44.072922 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:44.072840    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:44.210971 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:44.210946    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"]
Jul 28 10:35:44.212950 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:35:44.212925    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035a1d4f_39a7_4f52_94c3_2ac37fdaed68.slice/crio-27ae4aa73ca76991092b34c3db3de235c2e97a6271464bed57a9d8c0cc4daeb4 WatchSource:0}: Error finding container 27ae4aa73ca76991092b34c3db3de235c2e97a6271464bed57a9d8c0cc4daeb4: Status 404 returned error can't find the container with id 27ae4aa73ca76991092b34c3db3de235c2e97a6271464bed57a9d8c0cc4daeb4
Jul 28 10:35:44.625649 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:44.625611    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785" event={"ID":"035a1d4f-39a7-4f52-94c3-2ac37fdaed68","Type":"ContainerStarted","Data":"27ae4aa73ca76991092b34c3db3de235c2e97a6271464bed57a9d8c0cc4daeb4"}
Jul 28 10:35:46.837179 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:46.837140    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:35:46.837526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:46.837232    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:35:46.837526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:46.837278    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:35:47.641861 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:47.641826    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785" event={"ID":"035a1d4f-39a7-4f52-94c3-2ac37fdaed68","Type":"ContainerStarted","Data":"923c20a23807ae31b64432f2b15092f853a1d5f563fc03f7d61209d4b3b9ff5a"}
Jul 28 10:35:47.663165 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:47.663114    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785" podStartSLOduration=2.041184238 podStartE2EDuration="4.663100784s" totalImagesPullingTime="2.621916546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:35:47.662002228 +0000 UTC m=+772.939939403" watchObservedRunningTime="2026-07-28 10:35:47.663100784 +0000 UTC m=+772.941037957"
Jul 28 10:35:48.074014 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:48.073932    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:48.114825 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:48.114796    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:48.646720 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:48.646686    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:35:48.647787 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:35:48.647765    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-77785"
Jul 28 10:36:12.618226 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:12.618200    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5k7t4"
Jul 28 10:36:12.621307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:12.621291    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9whhf"
Jul 28 10:36:27.625847 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.625775    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:27.627988 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.627973    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:27.631075 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.631050    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 10:36:27.631832 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.631796    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-htdhd\""
Jul 28 10:36:27.631960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.631806    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 10:36:27.647911 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.647889    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:27.663826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.663804    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv7df\" (UniqueName: \"kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df\") pod \"kuadrant-operator-catalog-2rrpv\" (UID: \"5d25528b-a78b-44ca-90e1-de4e4e403e32\") " pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:27.764915 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.764891    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv7df\" (UniqueName: \"kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df\") pod \"kuadrant-operator-catalog-2rrpv\" (UID: \"5d25528b-a78b-44ca-90e1-de4e4e403e32\") " pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:27.773138 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.773117    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv7df\" (UniqueName: \"kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df\") pod \"kuadrant-operator-catalog-2rrpv\" (UID: \"5d25528b-a78b-44ca-90e1-de4e4e403e32\") " pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:27.938968 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.938900    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:27.992641 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:27.992613    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:28.076951 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.076823    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:28.079748 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:36:28.079719    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d25528b_a78b_44ca_90e1_de4e4e403e32.slice/crio-226e05c02755e32a75dd6c54ec0919be603e74c26be00e10ddfd728018bd9e7b WatchSource:0}: Error finding container 226e05c02755e32a75dd6c54ec0919be603e74c26be00e10ddfd728018bd9e7b: Status 404 returned error can't find the container with id 226e05c02755e32a75dd6c54ec0919be603e74c26be00e10ddfd728018bd9e7b
Jul 28 10:36:28.201599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.201503    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2jd9"]
Jul 28 10:36:28.204148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.204132    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:28.212951 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.212925    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2jd9"]
Jul 28 10:36:28.268910 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.268885    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxxgm\" (UniqueName: \"kubernetes.io/projected/66c8f60a-adc7-4198-bfa4-9f51eac3bf19-kube-api-access-qxxgm\") pod \"kuadrant-operator-catalog-b2jd9\" (UID: \"66c8f60a-adc7-4198-bfa4-9f51eac3bf19\") " pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:28.369470 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.369442    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxgm\" (UniqueName: \"kubernetes.io/projected/66c8f60a-adc7-4198-bfa4-9f51eac3bf19-kube-api-access-qxxgm\") pod \"kuadrant-operator-catalog-b2jd9\" (UID: \"66c8f60a-adc7-4198-bfa4-9f51eac3bf19\") " pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:28.377261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.377236    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxxgm\" (UniqueName: \"kubernetes.io/projected/66c8f60a-adc7-4198-bfa4-9f51eac3bf19-kube-api-access-qxxgm\") pod \"kuadrant-operator-catalog-b2jd9\" (UID: \"66c8f60a-adc7-4198-bfa4-9f51eac3bf19\") " pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:28.514626 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.514519    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:28.798905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.798880    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2jd9"]
Jul 28 10:36:28.817381 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:36:28.817355    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c8f60a_adc7_4198_bfa4_9f51eac3bf19.slice/crio-7e55a5ad125829a24b93146c74e67231e71fd4bc8bdadba04cb9292f7196a59c WatchSource:0}: Error finding container 7e55a5ad125829a24b93146c74e67231e71fd4bc8bdadba04cb9292f7196a59c: Status 404 returned error can't find the container with id 7e55a5ad125829a24b93146c74e67231e71fd4bc8bdadba04cb9292f7196a59c
Jul 28 10:36:28.823112 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:28.823080    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" event={"ID":"5d25528b-a78b-44ca-90e1-de4e4e403e32","Type":"ContainerStarted","Data":"226e05c02755e32a75dd6c54ec0919be603e74c26be00e10ddfd728018bd9e7b"}
Jul 28 10:36:29.830571 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:29.830522    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9" event={"ID":"66c8f60a-adc7-4198-bfa4-9f51eac3bf19","Type":"ContainerStarted","Data":"7e55a5ad125829a24b93146c74e67231e71fd4bc8bdadba04cb9292f7196a59c"}
Jul 28 10:36:30.836345 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:30.836310    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9" event={"ID":"66c8f60a-adc7-4198-bfa4-9f51eac3bf19","Type":"ContainerStarted","Data":"1ca629078ec1190656ae3be1647a913b4d135270fe78507153c88e6bb516d661"}
Jul 28 10:36:30.837735 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:30.837712    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" event={"ID":"5d25528b-a78b-44ca-90e1-de4e4e403e32","Type":"ContainerStarted","Data":"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"}
Jul 28 10:36:30.837874 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:30.837819    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" podUID="5d25528b-a78b-44ca-90e1-de4e4e403e32" containerName="registry-server" containerID="cri-o://5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4" gracePeriod=2
Jul 28 10:36:30.852507 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:30.852464    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9" podStartSLOduration=1.550637242 podStartE2EDuration="2.852447444s" totalImagesPullingTime="1.301810202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:36:30.850146369 +0000 UTC m=+816.128083545" watchObservedRunningTime="2026-07-28 10:36:30.852447444 +0000 UTC m=+816.130384618"
Jul 28 10:36:30.864884 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:30.864847    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" podStartSLOduration=1.827863481 podStartE2EDuration="3.864831245s" totalImagesPullingTime="2.036967764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:36:30.86427623 +0000 UTC m=+816.142213405" watchObservedRunningTime="2026-07-28 10:36:30.864831245 +0000 UTC m=+816.142768415"
Jul 28 10:36:31.084798 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.084773    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:31.195603 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.195542    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv7df\" (UniqueName: \"kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df\") pod \"5d25528b-a78b-44ca-90e1-de4e4e403e32\" (UID: \"5d25528b-a78b-44ca-90e1-de4e4e403e32\") "
Jul 28 10:36:31.197914 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.197885    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df" (OuterVolumeSpecName: "kube-api-access-hv7df") pod "5d25528b-a78b-44ca-90e1-de4e4e403e32" (UID: "5d25528b-a78b-44ca-90e1-de4e4e403e32"). InnerVolumeSpecName "kube-api-access-hv7df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:36:31.296422 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.296389    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv7df\" (UniqueName: \"kubernetes.io/projected/5d25528b-a78b-44ca-90e1-de4e4e403e32-kube-api-access-hv7df\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:36:31.843915 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.843883    2544 generic.go:356] "Generic (PLEG): container finished" podID="5d25528b-a78b-44ca-90e1-de4e4e403e32" containerID="5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4" exitCode=0
Jul 28 10:36:31.844374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.843927    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv"
Jul 28 10:36:31.844374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.843974    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" event={"ID":"5d25528b-a78b-44ca-90e1-de4e4e403e32","Type":"ContainerDied","Data":"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"}
Jul 28 10:36:31.844374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.844018    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2rrpv" event={"ID":"5d25528b-a78b-44ca-90e1-de4e4e403e32","Type":"ContainerDied","Data":"226e05c02755e32a75dd6c54ec0919be603e74c26be00e10ddfd728018bd9e7b"}
Jul 28 10:36:31.844374 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.844038    2544 scope.go:117] "RemoveContainer" containerID="5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"
Jul 28 10:36:31.853279 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.853262    2544 scope.go:117] "RemoveContainer" containerID="5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"
Jul 28 10:36:31.853641 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:36:31.853535    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4\": container with ID starting with 5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4 not found: ID does not exist" containerID="5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"
Jul 28 10:36:31.853641 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.853582    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4"} err="failed to get container status \"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4\": rpc error: code = NotFound desc = could not find container \"5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4\": container with ID starting with 5375ef872691446e5eeed1d1944bbbeb1c089238eda90d8550bcf572dddb70c4 not found: ID does not exist"
Jul 28 10:36:31.860443 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.860411    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:31.864028 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:31.864010    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2rrpv"]
Jul 28 10:36:33.269013 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:33.268979    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d25528b-a78b-44ca-90e1-de4e4e403e32" path="/var/lib/kubelet/pods/5d25528b-a78b-44ca-90e1-de4e4e403e32/volumes"
Jul 28 10:36:38.515528 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:38.515490    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:38.515955 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:38.515590    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:38.537740 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:38.537710    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:36:38.897032 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:36:38.897007    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-b2jd9"
Jul 28 10:37:01.671293 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.671252    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"]
Jul 28 10:37:01.671824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.671801    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d25528b-a78b-44ca-90e1-de4e4e403e32" containerName="registry-server"
Jul 28 10:37:01.671824 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.671822    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d25528b-a78b-44ca-90e1-de4e4e403e32" containerName="registry-server"
Jul 28 10:37:01.671940 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.671921    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d25528b-a78b-44ca-90e1-de4e4e403e32" containerName="registry-server"
Jul 28 10:37:01.676231 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.676210    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.679112 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.679094    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hxj26\""
Jul 28 10:37:01.692024 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.692003    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"]
Jul 28 10:37:01.750060 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.750022    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrllm\" (UniqueName: \"kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.750242 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.750102    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.850907 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.850869    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrllm\" (UniqueName: \"kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.851106 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.850968    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.851396 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.851370    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.859793 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.859766    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrllm\" (UniqueName: \"kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:01.986214 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:01.986127    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:02.124511 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:02.124488    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"]
Jul 28 10:37:02.126898 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:02.126874    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fed66e_8057_432a_bf63_3a38b1043108.slice/crio-c9d53983f13f6dcbf59ecaeb2c5b03fd3a472a74ac92bceebfa901d94552d36a WatchSource:0}: Error finding container c9d53983f13f6dcbf59ecaeb2c5b03fd3a472a74ac92bceebfa901d94552d36a: Status 404 returned error can't find the container with id c9d53983f13f6dcbf59ecaeb2c5b03fd3a472a74ac92bceebfa901d94552d36a
Jul 28 10:37:02.985515 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:02.985473    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf" event={"ID":"90fed66e-8057-432a-bf63-3a38b1043108","Type":"ContainerStarted","Data":"c9d53983f13f6dcbf59ecaeb2c5b03fd3a472a74ac92bceebfa901d94552d36a"}
Jul 28 10:37:06.239798 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.239763    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"]
Jul 28 10:37:06.247725 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.247704    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:06.251715 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.251687    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kdpq8\""
Jul 28 10:37:06.252761 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.252736    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"]
Jul 28 10:37:06.390895 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.390844    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgr7\" (UniqueName: \"kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7\") pod \"limitador-operator-controller-manager-859d7b78bd-46rwk\" (UID: \"182080c3-6a02-4cbe-8b5d-afeacfec0602\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:06.492478 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.492388    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgr7\" (UniqueName: \"kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7\") pod \"limitador-operator-controller-manager-859d7b78bd-46rwk\" (UID: \"182080c3-6a02-4cbe-8b5d-afeacfec0602\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:06.506872 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.506837    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgr7\" (UniqueName: \"kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7\") pod \"limitador-operator-controller-manager-859d7b78bd-46rwk\" (UID: \"182080c3-6a02-4cbe-8b5d-afeacfec0602\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:06.558700 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:06.558658    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:07.610792 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:07.610706    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"]
Jul 28 10:37:07.613091 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:07.613069    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182080c3_6a02_4cbe_8b5d_afeacfec0602.slice/crio-b3eb0e3be340bbea9f15c2b47ecc56c9d1f4c709d850fd024d3be1fdd9a45f12 WatchSource:0}: Error finding container b3eb0e3be340bbea9f15c2b47ecc56c9d1f4c709d850fd024d3be1fdd9a45f12: Status 404 returned error can't find the container with id b3eb0e3be340bbea9f15c2b47ecc56c9d1f4c709d850fd024d3be1fdd9a45f12
Jul 28 10:37:08.011162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:08.011121    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk" event={"ID":"182080c3-6a02-4cbe-8b5d-afeacfec0602","Type":"ContainerStarted","Data":"b3eb0e3be340bbea9f15c2b47ecc56c9d1f4c709d850fd024d3be1fdd9a45f12"}
Jul 28 10:37:08.012776 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:08.012749    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf" event={"ID":"90fed66e-8057-432a-bf63-3a38b1043108","Type":"ContainerStarted","Data":"0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d"}
Jul 28 10:37:08.012897 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:08.012809    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:08.039744 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:08.039695    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf" podStartSLOduration=1.6240745680000002 podStartE2EDuration="7.039676162s" totalImagesPullingTime="5.415601594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:08.038140243 +0000 UTC m=+853.316077417" watchObservedRunningTime="2026-07-28 10:37:08.039676162 +0000 UTC m=+853.317613335"
Jul 28 10:37:10.024060 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:10.024029    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk" event={"ID":"182080c3-6a02-4cbe-8b5d-afeacfec0602","Type":"ContainerStarted","Data":"80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2"}
Jul 28 10:37:10.024434 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:10.024140    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:10.048415 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:10.048374    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk" podStartSLOduration=2.512446116 podStartE2EDuration="4.048362359s" totalImagesPullingTime="1.535916243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:10.046430459 +0000 UTC m=+855.324367633" watchObservedRunningTime="2026-07-28 10:37:10.048362359 +0000 UTC m=+855.326299533"
Jul 28 10:37:19.020933 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:19.020904    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:20.729701 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.729665    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"]
Jul 28 10:37:20.730166 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.729884    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf" podUID="90fed66e-8057-432a-bf63-3a38b1043108" containerName="manager" containerID="cri-o://0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d" gracePeriod=2
Jul 28 10:37:20.737066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.737041    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"]
Jul 28 10:37:20.740300 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.740268    2544 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"90fed66e-8057-432a-bf63-3a38b1043108\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-5clcf\": pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:20.750920 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.750149    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"]
Jul 28 10:37:20.750920 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.750502    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerName="manager" containerID="cri-o://80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2" gracePeriod=2
Jul 28 10:37:20.754584 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.752757    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:20.756279 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.756253    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:20.756580 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.756430    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.36:8081/readyz\": EOF"
Jul 28 10:37:20.757165 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.757137    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90fed66e-8057-432a-bf63-3a38b1043108" containerName="manager"
Jul 28 10:37:20.757251 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.757168    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="90fed66e-8057-432a-bf63-3a38b1043108" containerName="manager"
Jul 28 10:37:20.757340 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.757325    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="90fed66e-8057-432a-bf63-3a38b1043108" containerName="manager"
Jul 28 10:37:20.761879 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.761203    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:20.762013 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.761907    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"]
Jul 28 10:37:20.769098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.769072    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:20.769751 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.769222    2544 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"182080c3-6a02-4cbe-8b5d-afeacfec0602\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-46rwk\": pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:20.774112 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.774081    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:20.777454 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.777431    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"]
Jul 28 10:37:20.777943 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.777928    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerName="manager"
Jul 28 10:37:20.777990 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.777948    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerName="manager"
Jul 28 10:37:20.778087 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.778076    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerName="manager"
Jul 28 10:37:20.781526 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.781509    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:20.793588 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.793534    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"]
Jul 28 10:37:20.797395 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.797368    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:20.799466 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.799443    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:20.821903 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:20.821875    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:20.911878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.911846    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r769s\" (UniqueName: \"kubernetes.io/projected/b5078561-31b2-4c06-96af-d1142fc8f1d2-kube-api-access-r769s\") pod \"limitador-operator-controller-manager-859d7b78bd-mh7qv\" (UID: \"b5078561-31b2-4c06-96af-d1142fc8f1d2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:20.912024 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.911893    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:20.912024 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:20.911984    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8cfx\" (UniqueName: \"kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.002184 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.002156    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.005119 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.005100    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.012604 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.012582    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8cfx\" (UniqueName: \"kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.012684 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.012641    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r769s\" (UniqueName: \"kubernetes.io/projected/b5078561-31b2-4c06-96af-d1142fc8f1d2-kube-api-access-r769s\") pod \"limitador-operator-controller-manager-859d7b78bd-mh7qv\" (UID: \"b5078561-31b2-4c06-96af-d1142fc8f1d2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:21.012684 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.012662    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.012959 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.012944    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.013011 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.012987    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.023650 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.023605    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.026006 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.025980    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.028116 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.028097    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.031199 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.031180    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:37:21.034494 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.034479    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.036829 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.036808    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.041010 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.040917    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8cfx\" (UniqueName: \"kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wdjhw\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.041010 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.040951    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r769s\" (UniqueName: \"kubernetes.io/projected/b5078561-31b2-4c06-96af-d1142fc8f1d2-kube-api-access-r769s\") pod \"limitador-operator-controller-manager-859d7b78bd-mh7qv\" (UID: \"b5078561-31b2-4c06-96af-d1142fc8f1d2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:21.061922 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.061898    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:37:21.077433 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.077409    2544 generic.go:356] "Generic (PLEG): container finished" podID="90fed66e-8057-432a-bf63-3a38b1043108" containerID="0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d" exitCode=0
Jul 28 10:37:21.077540 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.077467    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.077540 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.077510    2544 scope.go:117] "RemoveContainer" containerID="0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d"
Jul 28 10:37:21.078848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.078827    2544 generic.go:356] "Generic (PLEG): container finished" podID="182080c3-6a02-4cbe-8b5d-afeacfec0602" containerID="80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2" exitCode=0
Jul 28 10:37:21.078932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.078881    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.086472 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.086455    2544 scope.go:117] "RemoveContainer" containerID="0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d"
Jul 28 10:37:21.086743 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.086716    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d\": container with ID starting with 0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d not found: ID does not exist" containerID="0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d"
Jul 28 10:37:21.086789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.086750    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d"} err="failed to get container status \"0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d\": rpc error: code = NotFound desc = could not find container \"0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d\": container with ID starting with 0c4cc08bfc7cd21208e5690d9529b6d95316cebdca7286d52c7f3d3cbf5c0d5d not found: ID does not exist"
Jul 28 10:37:21.086789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.086774    2544 scope.go:117] "RemoveContainer" containerID="80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2"
Jul 28 10:37:21.094400 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.094379    2544 scope.go:117] "RemoveContainer" containerID="80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2"
Jul 28 10:37:21.094788 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.094769    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2\": container with ID starting with 80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2 not found: ID does not exist" containerID="80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2"
Jul 28 10:37:21.094870 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.094796    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2"} err="failed to get container status \"80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2\": rpc error: code = NotFound desc = could not find container \"80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2\": container with ID starting with 80ad88186cca5409f41c17f4ae7fca224d85f6d59addcd550dbece3d57339ce2 not found: ID does not exist"
Jul 28 10:37:21.097249 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.097223    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.099416 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.099393    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-46rwk\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-46rwk"
Jul 28 10:37:21.101441 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:21.101424    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5clcf\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="90fed66e-8057-432a-bf63-3a38b1043108" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5clcf"
Jul 28 10:37:21.113745 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.113723    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume\") pod \"90fed66e-8057-432a-bf63-3a38b1043108\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") "
Jul 28 10:37:21.113830 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.113786    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdgr7\" (UniqueName: \"kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7\") pod \"182080c3-6a02-4cbe-8b5d-afeacfec0602\" (UID: \"182080c3-6a02-4cbe-8b5d-afeacfec0602\") "
Jul 28 10:37:21.113868 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.113839    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrllm\" (UniqueName: \"kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm\") pod \"90fed66e-8057-432a-bf63-3a38b1043108\" (UID: \"90fed66e-8057-432a-bf63-3a38b1043108\") "
Jul 28 10:37:21.114214 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.114193    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "90fed66e-8057-432a-bf63-3a38b1043108" (UID: "90fed66e-8057-432a-bf63-3a38b1043108"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:37:21.115981 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.115961    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm" (OuterVolumeSpecName: "kube-api-access-nrllm") pod "90fed66e-8057-432a-bf63-3a38b1043108" (UID: "90fed66e-8057-432a-bf63-3a38b1043108"). InnerVolumeSpecName "kube-api-access-nrllm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:37:21.115981 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.115974    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7" (OuterVolumeSpecName: "kube-api-access-wdgr7") pod "182080c3-6a02-4cbe-8b5d-afeacfec0602" (UID: "182080c3-6a02-4cbe-8b5d-afeacfec0602"). InnerVolumeSpecName "kube-api-access-wdgr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:37:21.184426 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.184400    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:21.193868 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.193846    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:21.214605 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.214570    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.214767 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.214743    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvgpc\" (UniqueName: \"kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.214884 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.214858    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrllm\" (UniqueName: \"kubernetes.io/projected/90fed66e-8057-432a-bf63-3a38b1043108-kube-api-access-nrllm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:37:21.214884 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.214879    2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90fed66e-8057-432a-bf63-3a38b1043108-extensions-socket-volume\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:37:21.215015 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.214892    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdgr7\" (UniqueName: \"kubernetes.io/projected/182080c3-6a02-4cbe-8b5d-afeacfec0602-kube-api-access-wdgr7\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:37:21.277612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.277489    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182080c3-6a02-4cbe-8b5d-afeacfec0602" path="/var/lib/kubelet/pods/182080c3-6a02-4cbe-8b5d-afeacfec0602/volumes"
Jul 28 10:37:21.285682 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.280735    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90fed66e-8057-432a-bf63-3a38b1043108" path="/var/lib/kubelet/pods/90fed66e-8057-432a-bf63-3a38b1043108/volumes"
Jul 28 10:37:21.316071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.316047    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvgpc\" (UniqueName: \"kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.316195 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.316094    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.316514 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.316495    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.325475 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.325450    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvgpc\" (UniqueName: \"kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc\") pod \"kuadrant-operator-controller-manager-86878c769f-v9pd5\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.344101 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.344062    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:21.368174 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.368145    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:21.371438 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:21.371412    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5f6122_c3e2_4bcd_ba42_e407c1dcf7c6.slice/crio-d81a4467b169b04a22f46dc774c834e33b4500b5a3d6c8f912f1ff7bfa0a2783 WatchSource:0}: Error finding container d81a4467b169b04a22f46dc774c834e33b4500b5a3d6c8f912f1ff7bfa0a2783: Status 404 returned error can't find the container with id d81a4467b169b04a22f46dc774c834e33b4500b5a3d6c8f912f1ff7bfa0a2783
Jul 28 10:37:21.388237 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.388211    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"]
Jul 28 10:37:21.394087 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:21.394046    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5078561_31b2_4c06_96af_d1142fc8f1d2.slice/crio-1ed10ba9a8dc65523364757badbd0f91090628585e39119d02630fa6997f5b82 WatchSource:0}: Error finding container 1ed10ba9a8dc65523364757badbd0f91090628585e39119d02630fa6997f5b82: Status 404 returned error can't find the container with id 1ed10ba9a8dc65523364757badbd0f91090628585e39119d02630fa6997f5b82
Jul 28 10:37:21.520146 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:21.520123    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:37:21.522994 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:21.522970    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5fb345_91da_440a_87f0_3dce0cdd624e.slice/crio-8c203e8e815f62314d8e81210c6874a43ec2f32604253e4288796e9c63b03c0e WatchSource:0}: Error finding container 8c203e8e815f62314d8e81210c6874a43ec2f32604253e4288796e9c63b03c0e: Status 404 returned error can't find the container with id 8c203e8e815f62314d8e81210c6874a43ec2f32604253e4288796e9c63b03c0e
Jul 28 10:37:22.086682 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.086633    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" event={"ID":"4b5fb345-91da-440a-87f0-3dce0cdd624e","Type":"ContainerStarted","Data":"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"}
Jul 28 10:37:22.087123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.086695    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" event={"ID":"4b5fb345-91da-440a-87f0-3dce0cdd624e","Type":"ContainerStarted","Data":"8c203e8e815f62314d8e81210c6874a43ec2f32604253e4288796e9c63b03c0e"}
Jul 28 10:37:22.087123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.086815    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:22.089398 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.089361    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv" event={"ID":"b5078561-31b2-4c06-96af-d1142fc8f1d2","Type":"ContainerStarted","Data":"f394aba83f347b65b93354cfe263d5947bff9865e25de8c2963e59c97572c574"}
Jul 28 10:37:22.089545 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.089419    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv" event={"ID":"b5078561-31b2-4c06-96af-d1142fc8f1d2","Type":"ContainerStarted","Data":"1ed10ba9a8dc65523364757badbd0f91090628585e39119d02630fa6997f5b82"}
Jul 28 10:37:22.089911 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.089886    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:22.092846 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.092821    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" event={"ID":"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6","Type":"ContainerStarted","Data":"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"}
Jul 28 10:37:22.093020 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.092869    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" event={"ID":"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6","Type":"ContainerStarted","Data":"d81a4467b169b04a22f46dc774c834e33b4500b5a3d6c8f912f1ff7bfa0a2783"}
Jul 28 10:37:22.093101 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.093054    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:22.113878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.113837    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" podStartSLOduration=2.113823455 podStartE2EDuration="2.113823455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:22.111995979 +0000 UTC m=+867.389933153" watchObservedRunningTime="2026-07-28 10:37:22.113823455 +0000 UTC m=+867.391760698"
Jul 28 10:37:22.132348 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.132308    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" podStartSLOduration=2.132294248 podStartE2EDuration="2.132294248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:22.130716655 +0000 UTC m=+867.408653830" watchObservedRunningTime="2026-07-28 10:37:22.132294248 +0000 UTC m=+867.410231421"
Jul 28 10:37:22.147928 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:22.147889    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv" podStartSLOduration=2.147878675 podStartE2EDuration="2.147878675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:22.147418741 +0000 UTC m=+867.425355945" watchObservedRunningTime="2026-07-28 10:37:22.147878675 +0000 UTC m=+867.425815850"
Jul 28 10:37:33.105612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.105583    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:37:33.106074 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.105653    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:33.172510 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.172476    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:33.172693 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.172655    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" podUID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" containerName="manager" containerID="cri-o://d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845" gracePeriod=10
Jul 28 10:37:33.428411 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.428390    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:33.506899 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.506870    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8cfx\" (UniqueName: \"kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx\") pod \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") "
Jul 28 10:37:33.507056 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.506911    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume\") pod \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\" (UID: \"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6\") "
Jul 28 10:37:33.507314 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.507290    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" (UID: "9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:37:33.508945 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.508924    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx" (OuterVolumeSpecName: "kube-api-access-b8cfx") pod "9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" (UID: "9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6"). InnerVolumeSpecName "kube-api-access-b8cfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:37:33.607826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.607800    2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-extensions-socket-volume\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:37:33.607826 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:33.607822    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8cfx\" (UniqueName: \"kubernetes.io/projected/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6-kube-api-access-b8cfx\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:37:34.111060 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.111031    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mh7qv"
Jul 28 10:37:34.162650 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.162616    2544 generic.go:356] "Generic (PLEG): container finished" podID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" containerID="d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845" exitCode=0
Jul 28 10:37:34.162819 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.162681    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" event={"ID":"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6","Type":"ContainerDied","Data":"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"}
Jul 28 10:37:34.162819 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.162695    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"
Jul 28 10:37:34.162819 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.162724    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw" event={"ID":"9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6","Type":"ContainerDied","Data":"d81a4467b169b04a22f46dc774c834e33b4500b5a3d6c8f912f1ff7bfa0a2783"}
Jul 28 10:37:34.162819 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.162761    2544 scope.go:117] "RemoveContainer" containerID="d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"
Jul 28 10:37:34.173649 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.173496    2544 scope.go:117] "RemoveContainer" containerID="d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"
Jul 28 10:37:34.173821 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:37:34.173804    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845\": container with ID starting with d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845 not found: ID does not exist" containerID="d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"
Jul 28 10:37:34.173881 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.173830    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845"} err="failed to get container status \"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845\": rpc error: code = NotFound desc = could not find container \"d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845\": container with ID starting with d8799ee2d3d016dea68ae4288415cf79a8d66efd3e9ae3d12c48fc639a586845 not found: ID does not exist"
Jul 28 10:37:34.186891 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.186870    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:34.191424 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:34.191397    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wdjhw"]
Jul 28 10:37:35.270643 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:35.270612    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" path="/var/lib/kubelet/pods/9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6/volumes"
Jul 28 10:37:49.520179 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.520070    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"]
Jul 28 10:37:49.520541 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.520511    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" containerName="manager"
Jul 28 10:37:49.520541 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.520526    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" containerName="manager"
Jul 28 10:37:49.520683 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.520671    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5f6122-c3e2-4bcd-ba42-e407c1dcf7c6" containerName="manager"
Jul 28 10:37:49.523423 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.523404    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.527354 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.527331    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-88n8d\""
Jul 28 10:37:49.541317 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.541293    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"]
Jul 28 10:37:49.630441 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630403    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630444    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630474    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ec00fd-a31d-476d-baed-6719443b91fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630501    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630546    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630609    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ec00fd-a31d-476d-baed-6719443b91fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630638    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630656    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvrn\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-kube-api-access-6nvrn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.630749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.630698    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731674 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731635    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731722    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731739    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731767    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ec00fd-a31d-476d-baed-6719443b91fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731796    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.731848 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731847    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731876    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ec00fd-a31d-476d-baed-6719443b91fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731906    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732098 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.731930    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvrn\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-kube-api-access-6nvrn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732243 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.732222    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732338 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.732310    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732423 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.732399    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732710 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.732676    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.732887 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.732868    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ec00fd-a31d-476d-baed-6719443b91fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.734286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.734265    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ec00fd-a31d-476d-baed-6719443b91fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.734409 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.734392    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ec00fd-a31d-476d-baed-6719443b91fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.739497 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.739479    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.739790 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.739774    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvrn\" (UniqueName: \"kubernetes.io/projected/37ec00fd-a31d-476d-baed-6719443b91fb-kube-api-access-6nvrn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-628z8\" (UID: \"37ec00fd-a31d-476d-baed-6719443b91fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.834331 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.834273    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:49.976157 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.976120    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"]
Jul 28 10:37:49.981606 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:49.981574    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ec00fd_a31d_476d_baed_6719443b91fb.slice/crio-2473dee4e4f0b456b62742ad0b4c20e4aceca3f2f6f0185109a0782bc261bb34 WatchSource:0}: Error finding container 2473dee4e4f0b456b62742ad0b4c20e4aceca3f2f6f0185109a0782bc261bb34: Status 404 returned error can't find the container with id 2473dee4e4f0b456b62742ad0b4c20e4aceca3f2f6f0185109a0782bc261bb34
Jul 28 10:37:49.983631 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.983596    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:37:49.983721 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.983669    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:37:49.983721 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:49.983709    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:37:50.242685 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:50.242653    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" event={"ID":"37ec00fd-a31d-476d-baed-6719443b91fb","Type":"ContainerStarted","Data":"7eada6f6ab4ecad158682dde3ba095bd75a1a7ea0562cd902fe3c0088f57e985"}
Jul 28 10:37:50.243053 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:50.242704    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" event={"ID":"37ec00fd-a31d-476d-baed-6719443b91fb","Type":"ContainerStarted","Data":"2473dee4e4f0b456b62742ad0b4c20e4aceca3f2f6f0185109a0782bc261bb34"}
Jul 28 10:37:50.268177 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:50.268126    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" podStartSLOduration=1.268110905 podStartE2EDuration="1.268110905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:50.267500999 +0000 UTC m=+895.545438176" watchObservedRunningTime="2026-07-28 10:37:50.268110905 +0000 UTC m=+895.546048079"
Jul 28 10:37:50.834705 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:50.834677    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:51.594132 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.594101    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:37:51.597762 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.597742    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.603537 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.603515    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f66n2\""
Jul 28 10:37:51.603669 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.603586    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 10:37:51.608451 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.608423    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:37:51.638218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.638192    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:37:51.647684 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.647660    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-config-file\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.647768 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.647707    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dpb\" (UniqueName: \"kubernetes.io/projected/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-kube-api-access-r9dpb\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.749114 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.749086    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-config-file\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.749245 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.749144    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dpb\" (UniqueName: \"kubernetes.io/projected/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-kube-api-access-r9dpb\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.749808 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.749790    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-config-file\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.757789 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.757769    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dpb\" (UniqueName: \"kubernetes.io/projected/d9fa5e54-77d8-4fe8-903a-82ccf50e6a11-kube-api-access-r9dpb\") pod \"limitador-limitador-54d6d988d5-jmkpl\" (UID: \"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:51.839527 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.839503    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:51.910027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:51.909992    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:52.047548 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:52.047513    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:37:52.049549 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:37:52.049526    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fa5e54_77d8_4fe8_903a_82ccf50e6a11.slice/crio-4e643d6d9719c298194c5bcca3c7215380a2f55fb7be8cb15e2546a23bb2cc2f WatchSource:0}: Error finding container 4e643d6d9719c298194c5bcca3c7215380a2f55fb7be8cb15e2546a23bb2cc2f: Status 404 returned error can't find the container with id 4e643d6d9719c298194c5bcca3c7215380a2f55fb7be8cb15e2546a23bb2cc2f
Jul 28 10:37:52.254512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:52.254432    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl" event={"ID":"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11","Type":"ContainerStarted","Data":"4e643d6d9719c298194c5bcca3c7215380a2f55fb7be8cb15e2546a23bb2cc2f"}
Jul 28 10:37:52.254792 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:52.254766    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:52.255845 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:52.255824    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:37:55.200169 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:55.200138    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:37:55.200760 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:55.200734    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:37:55.272184 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:55.272153    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl" event={"ID":"d9fa5e54-77d8-4fe8-903a-82ccf50e6a11","Type":"ContainerStarted","Data":"1be87f775d4f76ab897eb31301e57e1cc34e9233204555abececee916fd1cf3b"}
Jul 28 10:37:55.272302 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:55.272291    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:37:55.287788 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:37:55.287741    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl" podStartSLOduration=1.856349374 podStartE2EDuration="4.287726952s" totalImagesPullingTime="2.431377578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:37:55.286540232 +0000 UTC m=+900.564477405" watchObservedRunningTime="2026-07-28 10:37:55.287726952 +0000 UTC m=+900.565664125"
Jul 28 10:38:06.277978 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:38:06.277952    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"
Jul 28 10:39:03.194766 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:03.194730    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:11.635003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.634973    2544 generic.go:356] "Generic (PLEG): container finished" podID="37ec00fd-a31d-476d-baed-6719443b91fb" containerID="7eada6f6ab4ecad158682dde3ba095bd75a1a7ea0562cd902fe3c0088f57e985" exitCode=0
Jul 28 10:39:11.635400 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.635048    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" event={"ID":"37ec00fd-a31d-476d-baed-6719443b91fb","Type":"ContainerDied","Data":"7eada6f6ab4ecad158682dde3ba095bd75a1a7ea0562cd902fe3c0088f57e985"}
Jul 28 10:39:11.635400 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.635388    2544 scope.go:117] "RemoveContainer" containerID="7eada6f6ab4ecad158682dde3ba095bd75a1a7ea0562cd902fe3c0088f57e985"
Jul 28 10:39:11.636333 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.636302    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:39:11.636443 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.636376    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:39:11.636443 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:11.636408    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:39:12.642313 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:12.642282    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" event={"ID":"37ec00fd-a31d-476d-baed-6719443b91fb","Type":"ContainerStarted","Data":"5ae759038295a36e44f41d8f85e1eedf17a304ddd422dc03f32f7d14a6964c4d"}
Jul 28 10:39:12.835320 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:12.835288    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:39:12.835513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:12.835419    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-628z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 10:39:12.835513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:12.835475    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" podUID="37ec00fd-a31d-476d-baed-6719443b91fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 10:39:13.834847 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:13.834810    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-628z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 10:39:13.835215 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:13.834866    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" podUID="37ec00fd-a31d-476d-baed-6719443b91fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 10:39:14.835268 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:14.835232    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-628z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 10:39:14.835653 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:14.835287    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" podUID="37ec00fd-a31d-476d-baed-6719443b91fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 10:39:15.835574 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:15.835523    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-628z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 10:39:15.835951 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:15.835600    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8" podUID="37ec00fd-a31d-476d-baed-6719443b91fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 10:39:16.838100 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:16.838019    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:39:16.838577 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:16.838404    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:39:16.838991 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:16.838974    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-628z8"
Jul 28 10:39:19.360201 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:19.360164    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:25.552521 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:25.552487    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:31.554886 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:31.554855    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:41.050035 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.049997    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"]
Jul 28 10:39:41.053894 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.053837    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.058153 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.058131    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-65s9d\""
Jul 28 10:39:41.058277 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.058134    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 10:39:41.058277 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.058134    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 10:39:41.058277 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.058259    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 10:39:41.064716 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.064695    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"]
Jul 28 10:39:41.165898 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.165861    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a2bb213-c041-45ae-8311-46380a9cd3a9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.166064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.165919    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.166064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.165944    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nggs7\" (UniqueName: \"kubernetes.io/projected/1a2bb213-c041-45ae-8311-46380a9cd3a9-kube-api-access-nggs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.166064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.165969    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.166064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.166003    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.166064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.166050    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266512 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266479    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a2bb213-c041-45ae-8311-46380a9cd3a9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266529    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266551    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nggs7\" (UniqueName: \"kubernetes.io/projected/1a2bb213-c041-45ae-8311-46380a9cd3a9-kube-api-access-nggs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266598    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266624    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.266706 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.266646    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.267064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.267029    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.267064 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.267059    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.267216 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.267075    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.269123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.269103    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a2bb213-c041-45ae-8311-46380a9cd3a9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.269430 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.269413    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a2bb213-c041-45ae-8311-46380a9cd3a9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.274856 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.274833    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nggs7\" (UniqueName: \"kubernetes.io/projected/1a2bb213-c041-45ae-8311-46380a9cd3a9-kube-api-access-nggs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m\" (UID: \"1a2bb213-c041-45ae-8311-46380a9cd3a9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.365741 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.365709    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:41.509455 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.509428    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"]
Jul 28 10:39:41.511835 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:39:41.511809    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2bb213_c041_45ae_8311_46380a9cd3a9.slice/crio-90aaba3938e4426ab85e36fd8a5bba77838f352f7ab2d89499f62ab6026e2162 WatchSource:0}: Error finding container 90aaba3938e4426ab85e36fd8a5bba77838f352f7ab2d89499f62ab6026e2162: Status 404 returned error can't find the container with id 90aaba3938e4426ab85e36fd8a5bba77838f352f7ab2d89499f62ab6026e2162
Jul 28 10:39:41.780930 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:41.780852    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m" event={"ID":"1a2bb213-c041-45ae-8311-46380a9cd3a9","Type":"ContainerStarted","Data":"90aaba3938e4426ab85e36fd8a5bba77838f352f7ab2d89499f62ab6026e2162"}
Jul 28 10:39:42.788043 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:42.788003    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m" event={"ID":"1a2bb213-c041-45ae-8311-46380a9cd3a9","Type":"ContainerStarted","Data":"49d0902b8492545605a60f96e4fbf7be91832f97f90c9bfc24658f9b4234d839"}
Jul 28 10:39:42.806073 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:42.806026    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m" podStartSLOduration=0.72364575 podStartE2EDuration="1.806011354s" totalImagesPullingTime="1.082365604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:39:42.805550088 +0000 UTC m=+1008.083487263" watchObservedRunningTime="2026-07-28 10:39:42.806011354 +0000 UTC m=+1008.083948525"
Jul 28 10:39:42.951612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:42.951578    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:51.366730 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:51.366694    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:51.367211 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:51.366767    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:51.379024 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:51.379001    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:51.841419 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:51.841350    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m"
Jul 28 10:39:53.746450 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:53.746416    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:39:58.129027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.128999    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"]
Jul 28 10:39:58.133155 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.133132    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.135758 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.135740    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 10:39:58.142453 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.142432    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"]
Jul 28 10:39:58.216857 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.216830    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.217000 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.216864    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.217000 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.216890    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjhjs\" (UniqueName: \"kubernetes.io/projected/c99e0cf6-ce79-4e50-a8d8-afae068c7114-kube-api-access-zjhjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.217000 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.216989    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.217124 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.217024    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.217124 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.217086    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318366 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318337    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318391    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318408    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318429    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjhjs\" (UniqueName: \"kubernetes.io/projected/c99e0cf6-ce79-4e50-a8d8-afae068c7114-kube-api-access-zjhjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318490    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318704 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318518    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318820 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318799    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318868 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318816    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.318962 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.318944    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.320879 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.320852    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99e0cf6-ce79-4e50-a8d8-afae068c7114-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.321116 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.321099    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e0cf6-ce79-4e50-a8d8-afae068c7114-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.325974 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.325953    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjhjs\" (UniqueName: \"kubernetes.io/projected/c99e0cf6-ce79-4e50-a8d8-afae068c7114-kube-api-access-zjhjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7\" (UID: \"c99e0cf6-ce79-4e50-a8d8-afae068c7114\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.443774 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.443699    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:39:58.578585 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.578544    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"]
Jul 28 10:39:58.580436 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:39:58.580412    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc99e0cf6_ce79_4e50_a8d8_afae068c7114.slice/crio-933867d8bdbd9e7360c307a4d9f9c14778f0cb50335867ea09976b7273cf4602 WatchSource:0}: Error finding container 933867d8bdbd9e7360c307a4d9f9c14778f0cb50335867ea09976b7273cf4602: Status 404 returned error can't find the container with id 933867d8bdbd9e7360c307a4d9f9c14778f0cb50335867ea09976b7273cf4602
Jul 28 10:39:58.864677 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.864641    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7" event={"ID":"c99e0cf6-ce79-4e50-a8d8-afae068c7114","Type":"ContainerStarted","Data":"117422abf88e17c99f20086c8b4670e0b26db79e90c422360b839616a75571c8"}
Jul 28 10:39:58.864677 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.864681    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7" event={"ID":"c99e0cf6-ce79-4e50-a8d8-afae068c7114","Type":"ContainerStarted","Data":"933867d8bdbd9e7360c307a4d9f9c14778f0cb50335867ea09976b7273cf4602"}
Jul 28 10:39:58.882329 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:39:58.882278    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7" podStartSLOduration=0.70994881 podStartE2EDuration="882.257912ms" totalImagesPullingTime="172.309102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:39:58.881288676 +0000 UTC m=+1024.159225870" watchObservedRunningTime="2026-07-28 10:39:58.882257912 +0000 UTC m=+1024.160195087"
Jul 28 10:40:07.745397 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:40:07.745363    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:40:08.444316 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:40:08.444283    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:40:08.444491 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:40:08.444326    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:40:08.456884 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:40:08.456862    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:40:08.925419 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:40:08.925395    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7"
Jul 28 10:41:42.353290 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:41:42.353256    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:41:52.244471 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:41:52.244437    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:42:03.754034 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:42:03.753993    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:42:55.230269 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:42:55.230241    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:42:55.231682 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:42:55.231661    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:43:45.741803 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:43:45.741769    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:43:55.384713 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:43:55.384629    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:44:24.344140 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:44:24.344108    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:44:34.445711 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:44:34.445677    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:44:43.146847 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:44:43.146811    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:44:52.447917 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:44:52.447882    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:45:02.743301 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:45:02.743265    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:45:11.950180 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:45:11.950141    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:45:24.546421 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:45:24.546340    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:45:35.148637 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:45:35.148605    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:45:56.148694 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:45:56.148658    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:46:06.942897 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:46:06.942864    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:46:29.850804 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:46:29.850770    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:46:42.053943 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:46:42.053900    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:46:52.647955 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:46:52.647872    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:09.967347 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:09.967309    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:18.949370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:18.949339    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:30.780049 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:30.780011    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:40.457229 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:40.457194    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:49.953600 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:49.953550    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:47:55.261441 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:55.261415    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:47:55.263239 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:47:55.263216    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:48:18.267785 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:18.267703    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:48:18.268266 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:18.267921    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" podUID="4b5fb345-91da-440a-87f0-3dce0cdd624e" containerName="manager" containerID="cri-o://9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd" gracePeriod=10
Jul 28 10:48:22.220212 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.220188    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:48:22.331370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.331287    2544 generic.go:356] "Generic (PLEG): container finished" podID="4b5fb345-91da-440a-87f0-3dce0cdd624e" containerID="9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd" exitCode=0
Jul 28 10:48:22.331370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.331329    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" event={"ID":"4b5fb345-91da-440a-87f0-3dce0cdd624e","Type":"ContainerDied","Data":"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"}
Jul 28 10:48:22.331370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.331333    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"
Jul 28 10:48:22.331370 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.331371    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5" event={"ID":"4b5fb345-91da-440a-87f0-3dce0cdd624e","Type":"ContainerDied","Data":"8c203e8e815f62314d8e81210c6874a43ec2f32604253e4288796e9c63b03c0e"}
Jul 28 10:48:22.331748 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.331393    2544 scope.go:117] "RemoveContainer" containerID="9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"
Jul 28 10:48:22.337307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.337288    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume\") pod \"4b5fb345-91da-440a-87f0-3dce0cdd624e\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") "
Jul 28 10:48:22.337411 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.337379    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvgpc\" (UniqueName: \"kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc\") pod \"4b5fb345-91da-440a-87f0-3dce0cdd624e\" (UID: \"4b5fb345-91da-440a-87f0-3dce0cdd624e\") "
Jul 28 10:48:22.337652 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.337629    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b5fb345-91da-440a-87f0-3dce0cdd624e" (UID: "4b5fb345-91da-440a-87f0-3dce0cdd624e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:48:22.339452 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.339427    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc" (OuterVolumeSpecName: "kube-api-access-cvgpc") pod "4b5fb345-91da-440a-87f0-3dce0cdd624e" (UID: "4b5fb345-91da-440a-87f0-3dce0cdd624e"). InnerVolumeSpecName "kube-api-access-cvgpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:48:22.341701 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.341679    2544 scope.go:117] "RemoveContainer" containerID="9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"
Jul 28 10:48:22.341949 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:48:22.341931    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd\": container with ID starting with 9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd not found: ID does not exist" containerID="9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"
Jul 28 10:48:22.342022 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.341960    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd"} err="failed to get container status \"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd\": rpc error: code = NotFound desc = could not find container \"9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd\": container with ID starting with 9058e0baee2aae66f1022d1959eb05dae10968893fd8e9b91624e93d847846fd not found: ID does not exist"
Jul 28 10:48:22.438905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.438862    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvgpc\" (UniqueName: \"kubernetes.io/projected/4b5fb345-91da-440a-87f0-3dce0cdd624e-kube-api-access-cvgpc\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:48:22.438905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.438894    2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b5fb345-91da-440a-87f0-3dce0cdd624e-extensions-socket-volume\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:48:22.656437 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.656410    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:48:22.662650 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:22.662625    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v9pd5"]
Jul 28 10:48:23.269407 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:48:23.269380    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b5fb345-91da-440a-87f0-3dce0cdd624e" path="/var/lib/kubelet/pods/4b5fb345-91da-440a-87f0-3dce0cdd624e/volumes"
Jul 28 10:49:19.408361 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.408327    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"]
Jul 28 10:49:19.408813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.408705    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5fb345-91da-440a-87f0-3dce0cdd624e" containerName="manager"
Jul 28 10:49:19.408813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.408719    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5fb345-91da-440a-87f0-3dce0cdd624e" containerName="manager"
Jul 28 10:49:19.408813 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.408791    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5fb345-91da-440a-87f0-3dce0cdd624e" containerName="manager"
Jul 28 10:49:19.413010 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.412987    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.415905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.415888    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hxj26\""
Jul 28 10:49:19.425103 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.425081    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"]
Jul 28 10:49:19.427147 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.427123    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jgvl\" (UniqueName: \"kubernetes.io/projected/5a98b69b-fb68-4085-8857-c1b6bb665ea5-kube-api-access-5jgvl\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.427262 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.427182    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a98b69b-fb68-4085-8857-c1b6bb665ea5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.528322 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.528292    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a98b69b-fb68-4085-8857-c1b6bb665ea5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.528503 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.528371    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jgvl\" (UniqueName: \"kubernetes.io/projected/5a98b69b-fb68-4085-8857-c1b6bb665ea5-kube-api-access-5jgvl\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.528737 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.528716    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a98b69b-fb68-4085-8857-c1b6bb665ea5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.540903 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.540883    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jgvl\" (UniqueName: \"kubernetes.io/projected/5a98b69b-fb68-4085-8857-c1b6bb665ea5-kube-api-access-5jgvl\") pod \"kuadrant-operator-controller-manager-86878c769f-c8lc9\" (UID: \"5a98b69b-fb68-4085-8857-c1b6bb665ea5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.724343 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.724248    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:19.864382 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:19.864353    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"]
Jul 28 10:49:19.865943 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:49:19.865916    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a98b69b_fb68_4085_8857_c1b6bb665ea5.slice/crio-b7bae3c236450a18e38d0e07c0331431f3e4618ae1213282870e2afcccfd47e0 WatchSource:0}: Error finding container b7bae3c236450a18e38d0e07c0331431f3e4618ae1213282870e2afcccfd47e0: Status 404 returned error can't find the container with id b7bae3c236450a18e38d0e07c0331431f3e4618ae1213282870e2afcccfd47e0
Jul 28 10:49:20.621363 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:20.621331    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9" event={"ID":"5a98b69b-fb68-4085-8857-c1b6bb665ea5","Type":"ContainerStarted","Data":"963d057b777d46a105f69d29c1c9ef1e6a27e1051dc000086d632d05dd47483c"}
Jul 28 10:49:20.621749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:20.621395    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:20.621749 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:20.621417    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9" event={"ID":"5a98b69b-fb68-4085-8857-c1b6bb665ea5","Type":"ContainerStarted","Data":"b7bae3c236450a18e38d0e07c0331431f3e4618ae1213282870e2afcccfd47e0"}
Jul 28 10:49:20.644093 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:20.644044    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9" podStartSLOduration=1.6440305149999999 podStartE2EDuration="1.644030515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:49:20.641062201 +0000 UTC m=+1585.918999375" watchObservedRunningTime="2026-07-28 10:49:20.644030515 +0000 UTC m=+1585.921967689"
Jul 28 10:49:31.629437 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:31.629402    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8lc9"
Jul 28 10:49:41.454123 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:41.454085    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:49:52.148137 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:49:52.148061    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:50:06.954834 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:50:06.954796    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:50:18.678707 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:50:18.678664    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:50:29.144416 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:50:29.144381    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:50:38.553016 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:50:38.552983    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:50:48.844164 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:50:48.844122    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:51:00.641808 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:51:00.641769    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:51:08.552057 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:51:08.552028    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:51:18.963832 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:51:18.963625    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:51:32.756040 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:51:32.756000    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:51:41.957202 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:51:41.957169    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:52:02.104247 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:02.104211    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:52:09.657378 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:09.657346    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:52:21.552862 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:21.552825    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:52:55.297788 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:55.297754    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:52:55.300905 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:55.300882    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:52:58.554770 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.554734    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:52:58.558352 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.558332    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.560817 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.560797    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6215f6af-kserve-self-signed-certs\""
Jul 28 10:52:58.568881 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.568860    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:52:58.647390 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647367    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.647502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647401    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdq6c\" (UniqueName: \"kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.647502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647430    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.647502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647484    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.647685 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647545    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.647685 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.647617    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749003 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.748974    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749016    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749052    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749075    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdq6c\" (UniqueName: \"kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749093    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749129    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749617 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749591    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749707 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749613    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.749707 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.749646    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.751458 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.751439    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.751670 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.751653    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.757686 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.757668    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdq6c\" (UniqueName: \"kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c\") pod \"e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") " pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:58.867638 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:58.867614    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:52:59.007077 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:59.007051    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:52:59.009731 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:52:59.009703    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75fd19ea_bd78_418d_9c0b_aaa15cbb445d.slice/crio-5a147be69fd7dadfc9455c2696f12a9e25f688e4e4d49b5dc9b685f1f1362b57 WatchSource:0}: Error finding container 5a147be69fd7dadfc9455c2696f12a9e25f688e4e4d49b5dc9b685f1f1362b57: Status 404 returned error can't find the container with id 5a147be69fd7dadfc9455c2696f12a9e25f688e4e4d49b5dc9b685f1f1362b57
Jul 28 10:52:59.690762 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:59.690720    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" event={"ID":"75fd19ea-bd78-418d-9c0b-aaa15cbb445d","Type":"ContainerStarted","Data":"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"}
Jul 28 10:52:59.691218 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:59.690775    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" event={"ID":"75fd19ea-bd78-418d-9c0b-aaa15cbb445d","Type":"ContainerStarted","Data":"5a147be69fd7dadfc9455c2696f12a9e25f688e4e4d49b5dc9b685f1f1362b57"}
Jul 28 10:52:59.711052 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:52:59.711006    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" podStartSLOduration=1.7109917700000001 podStartE2EDuration="1.71099177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:52:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:52:59.709494227 +0000 UTC m=+1804.987431401" watchObservedRunningTime="2026-07-28 10:52:59.71099177 +0000 UTC m=+1804.988928943"
Jul 28 10:53:02.356863 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:02.356825    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:53:08.868425 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:08.868388    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:08.868425 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:08.868432    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:08.880909 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:08.880885    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:09.751533 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:09.751510    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:40.865041 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:40.865008    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:53:43.244973 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:43.244930    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:53:55.884970 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:55.884907    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" podUID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" containerName="main" containerID="cri-o://576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f" gracePeriod=45
Jul 28 10:53:56.150933 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.150862    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:56.260874 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.260824    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.260896    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.260938    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.260960    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdq6c\" (UniqueName: \"kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.260996    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261080 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.261036    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs\") pod \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\" (UID: \"75fd19ea-bd78-418d-9c0b-aaa15cbb445d\") "
Jul 28 10:53:56.261330 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.261242    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home" (OuterVolumeSpecName: "home") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:53:56.261330 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.261228    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:53:56.261330 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.261307    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache" (OuterVolumeSpecName: "model-cache") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:53:56.263326 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.263302    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm" (OuterVolumeSpecName: "dshm") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:53:56.263428 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.263325    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c" (OuterVolumeSpecName: "kube-api-access-vdq6c") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "kube-api-access-vdq6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:53:56.263428 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.263344    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75fd19ea-bd78-418d-9c0b-aaa15cbb445d" (UID: "75fd19ea-bd78-418d-9c0b-aaa15cbb445d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:53:56.362148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362111    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tmp-dir\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.362148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362141    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdq6c\" (UniqueName: \"kubernetes.io/projected/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-kube-api-access-vdq6c\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.362148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362153    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-model-cache\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.362148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362161    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-tls-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.362426 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362169    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-dshm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.362426 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.362177    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75fd19ea-bd78-418d-9c0b-aaa15cbb445d-home\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:53:56.746949 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.746920    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:53:56.983759 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.983719    2544 generic.go:356] "Generic (PLEG): container finished" podID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" containerID="576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f" exitCode=0
Jul 28 10:53:56.984148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.983764    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" event={"ID":"75fd19ea-bd78-418d-9c0b-aaa15cbb445d","Type":"ContainerDied","Data":"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"}
Jul 28 10:53:56.984148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.983812    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl" event={"ID":"75fd19ea-bd78-418d-9c0b-aaa15cbb445d","Type":"ContainerDied","Data":"5a147be69fd7dadfc9455c2696f12a9e25f688e4e4d49b5dc9b685f1f1362b57"}
Jul 28 10:53:56.984148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.983831    2544 scope.go:117] "RemoveContainer" containerID="576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"
Jul 28 10:53:56.984148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.983775    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"
Jul 28 10:53:56.993954 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.993934    2544 scope.go:117] "RemoveContainer" containerID="576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"
Jul 28 10:53:56.994280 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:53:56.994258    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f\": container with ID starting with 576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f not found: ID does not exist" containerID="576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"
Jul 28 10:53:56.994375 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:56.994287    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f"} err="failed to get container status \"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f\": rpc error: code = NotFound desc = could not find container \"576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f\": container with ID starting with 576157f5af6d910a81e79ea39be99c9690cdba75e394bf3061acd987d5ee2f3f not found: ID does not exist"
Jul 28 10:53:57.010086 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:57.010018    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:53:57.013803 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:57.013780    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6215f6af-kserve-5bfcb78559-wp7vl"]
Jul 28 10:53:57.273399 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:53:57.273322    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" path="/var/lib/kubelet/pods/75fd19ea-bd78-418d-9c0b-aaa15cbb445d/volumes"
Jul 28 10:54:06.658871 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:06.658828    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:54:18.344488 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:18.344406    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:54:28.946077 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:28.945990    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:54:38.248411 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:38.248381    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:54:47.355307 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:47.355274    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:54:57.847677 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:54:57.847637    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:55:07.345815 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:55:07.345781    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:55:20.655311 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:55:20.655279    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:55:30.058659 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:55:30.058624    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:55:39.477758 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:55:39.477726    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:55:51.353067 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:55:51.352994    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:02.149665 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:02.149625    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:12.352786 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:12.352746    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:23.758117 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:23.758080    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:36.166800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:36.166763    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:46.658603 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:46.658567    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:56:54.481726 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:56:54.481696    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:57:03.985768 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:03.985730    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:57:25.459465 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:25.459383    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:57:33.283800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.283760    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:57:33.284256 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.284179    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" containerName="main"
Jul 28 10:57:33.284256 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.284191    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" containerName="main"
Jul 28 10:57:33.284368 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.284264    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="75fd19ea-bd78-418d-9c0b-aaa15cbb445d" containerName="main"
Jul 28 10:57:33.287323 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.287307    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.291200 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.291178    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3eefdb25-openshift-default-dockercfg-ng8rw\""
Jul 28 10:57:33.317816 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.317794    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:57:33.341643 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341615    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341671    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341716    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341742    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341788    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341804    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvxz8\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341841    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.341936 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341857    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.342096 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.341946    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442608 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442574    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442616    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442634    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442666    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442681    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvxz8\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442724    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442741    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.442800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442768    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.442828    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443175 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.443049    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443377 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.443353    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443460 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.443377    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443460 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.443448    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.443968 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.443902    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.445236 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.445216    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.445768 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.445748    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.455518 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.455495    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.455612 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.455527    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvxz8\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8\") pod \"e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") " pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.598914 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.598839    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:33.755629 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.755605    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:57:33.757325 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:57:33.757298    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode748513b_e8a8_4734_b419_3a4bf1e3a0d3.slice/crio-ff1371cf19c35b4834f31ab1885ae6e401c5b86cac74fe5d5cdd925dd7fbbff7 WatchSource:0}: Error finding container ff1371cf19c35b4834f31ab1885ae6e401c5b86cac74fe5d5cdd925dd7fbbff7: Status 404 returned error can't find the container with id ff1371cf19c35b4834f31ab1885ae6e401c5b86cac74fe5d5cdd925dd7fbbff7
Jul 28 10:57:33.759232 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.759200    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:33.759311 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.759268    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:33.759311 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:33.759296    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:34.074296 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:34.074258    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerStarted","Data":"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"}
Jul 28 10:57:34.074460 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:34.074320    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerStarted","Data":"ff1371cf19c35b4834f31ab1885ae6e401c5b86cac74fe5d5cdd925dd7fbbff7"}
Jul 28 10:57:34.095836 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:34.095778    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podStartSLOduration=1.095758556 podStartE2EDuration="1.095758556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:57:34.094387514 +0000 UTC m=+2079.372324691" watchObservedRunningTime="2026-07-28 10:57:34.095758556 +0000 UTC m=+2079.373695730"
Jul 28 10:57:34.599060 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:34.599026    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:34.604773 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:34.604747    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:35.079568 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:35.079535    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:35.080524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:35.080507    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:41.843639 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:41.843601    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:57:50.081150 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.081058    2544 patch_prober.go:28] interesting pod/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 10:57:50.081150 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.081127    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 10:57:50.161501 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.161411    2544 generic.go:356] "Generic (PLEG): container finished" podID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerID="bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9" exitCode=0
Jul 28 10:57:50.161670 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.161493    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerDied","Data":"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"}
Jul 28 10:57:50.161932 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.161919    2544 scope.go:117] "RemoveContainer" containerID="bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"
Jul 28 10:57:50.162952 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.162920    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:50.163006 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.162995    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:50.163043 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:50.163031    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:57:51.169293 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:51.169262    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerStarted","Data":"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"}
Jul 28 10:57:51.599957 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:51.599850    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:51.600269 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:51.599916    2544 patch_prober.go:28] interesting pod/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 10:57:51.600502 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:51.600464    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 10:57:52.600242 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:52.599813    2544 patch_prober.go:28] interesting pod/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 10:57:52.600242 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:52.599870    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 10:57:53.600237 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:53.600201    2544 patch_prober.go:28] interesting pod/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 10:57:53.600535 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:53.600495    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 10:57:54.599377 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:54.599331    2544 patch_prober.go:28] interesting pod/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 10:57:54.599617 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:54.599408    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 10:57:55.332134 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:55.332103    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:57:55.335480 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:55.335458    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 10:57:55.603456 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:55.603428    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:55.603896 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:55.603881    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:57:55.604480 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:57:55.604460    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:58:14.443607 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:14.443573    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:58:16.456464 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:16.456423    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:58:16.456893 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:16.456695    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy" containerID="cri-o://c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271" gracePeriod=30
Jul 28 10:58:21.474330 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:58:21.473963    2544 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode748513b_e8a8_4734_b419_3a4bf1e3a0d3.slice/crio-c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271.scope/cpu.weight": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode748513b_e8a8_4734_b419_3a4bf1e3a0d3.slice/crio-c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271.scope/cpu.weight: no such device
Jul 28 10:58:21.729302 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.729241    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:58:21.868330 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868303    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868330 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868333    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868366    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvxz8\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868392    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868428    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868473    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868524 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868507    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868810 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868585    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868810 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868586    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:58:21.868810 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868630    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket\") pod \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\" (UID: \"e748513b-e8a8-4734-b419-3a4bf1e3a0d3\") "
Jul 28 10:58:21.868810 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868766    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:58:21.869012 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868873    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 10:58:21.869012 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.868957    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:58:21.869118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.869026    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.869118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.869046    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.869118 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.869061    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.869426 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.869235    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:58:21.870947 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.870917    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:58:21.871066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.870961    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8" (OuterVolumeSpecName: "kube-api-access-dvxz8") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "kube-api-access-dvxz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:58:21.871066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.870966    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:58:21.871066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.871055    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e748513b-e8a8-4734-b419-3a4bf1e3a0d3" (UID: "e748513b-e8a8-4734-b419-3a4bf1e3a0d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 10:58:21.970071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970051    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.970071 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970072    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvxz8\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-kube-api-access-dvxz8\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.970224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970082    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.970224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970090    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.970224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970097    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:21.970224 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:21.970104    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e748513b-e8a8-4734-b419-3a4bf1e3a0d3-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 10:58:22.330800 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.330767    2544 generic.go:356] "Generic (PLEG): container finished" podID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerID="c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271" exitCode=0
Jul 28 10:58:22.330960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.330815    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerDied","Data":"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"}
Jul 28 10:58:22.330960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.330831    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"
Jul 28 10:58:22.330960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.330865    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt" event={"ID":"e748513b-e8a8-4734-b419-3a4bf1e3a0d3","Type":"ContainerDied","Data":"ff1371cf19c35b4834f31ab1885ae6e401c5b86cac74fe5d5cdd925dd7fbbff7"}
Jul 28 10:58:22.330960 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.330894    2544 scope.go:117] "RemoveContainer" containerID="c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"
Jul 28 10:58:22.348261 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.348238    2544 scope.go:117] "RemoveContainer" containerID="bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"
Jul 28 10:58:22.356482 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.356460    2544 scope.go:117] "RemoveContainer" containerID="c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"
Jul 28 10:58:22.356907 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:58:22.356877    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271\": container with ID starting with c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271 not found: ID does not exist" containerID="c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"
Jul 28 10:58:22.356907 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.356917    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271"} err="failed to get container status \"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271\": rpc error: code = NotFound desc = could not find container \"c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271\": container with ID starting with c5ca1d35f7ed204250796244ce51b669aceb6e4751a9f3b525388ab0ff3d8271 not found: ID does not exist"
Jul 28 10:58:22.357190 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.356940    2544 scope.go:117] "RemoveContainer" containerID="bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"
Jul 28 10:58:22.357190 ip-10-0-129-31 kubenswrapper[2544]: E0728 10:58:22.357166    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9\": container with ID starting with bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9 not found: ID does not exist" containerID="bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"
Jul 28 10:58:22.357319 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.357187    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9"} err="failed to get container status \"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9\": rpc error: code = NotFound desc = could not find container \"bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9\": container with ID starting with bd2e6f41ff9fc28536577647fe59d3628dad5fe9143f9bde977ed982580fe4f9 not found: ID does not exist"
Jul 28 10:58:22.359442 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.359420    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:58:22.361198 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:22.361178    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3eefdb25-openshift-default-78bdbf9c64-jjzrt"]
Jul 28 10:58:23.269793 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:23.269757    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" path="/var/lib/kubelet/pods/e748513b-e8a8-4734-b419-3a4bf1e3a0d3/volumes"
Jul 28 10:58:37.648205 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:58:37.648165    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:59:11.646473 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:11.646435    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:59:14.417941 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.417899    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 10:59:14.418517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.418486    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 10:59:14.418517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.418512    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 10:59:14.418697 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.418636    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 10:59:14.422022 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.422001    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.431027 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.430757    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4217be61-openshift-default-dockercfg-nzrsd\""
Jul 28 10:59:14.435513 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.435485    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 10:59:14.537869 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.537829    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.537869 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.537872    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.537895    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.537956    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.537996    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538066 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.538038    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538199 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.538116    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538199 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.538156    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.538199 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.538172    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbfll\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.638658 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638625    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.638873 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638667    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.638873 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638689    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.638873 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638818    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.638873 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638867    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639095 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638925    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639095 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.638982    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639095 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639033    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639095 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639062    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbfll\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639298 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639104    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639298 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639169    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639574    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639599 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639594    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.639762 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.639631    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.641233 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.641214    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.641940 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.641919    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.647843 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.647815    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.647929 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.647866    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbfll\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll\") pod \"e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") " pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.754804 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.754732    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:14.898878 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.898852    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 10:59:14.900709 ip-10-0-129-31 kubenswrapper[2544]: W0728 10:59:14.900680    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4138de3_5b47_4965_95b7_1957b019bdbc.slice/crio-c2d99c5a3c0befd38a1cddab6b792b52452d248dc0fc0797971657fbffdd69e0 WatchSource:0}: Error finding container c2d99c5a3c0befd38a1cddab6b792b52452d248dc0fc0797971657fbffdd69e0: Status 404 returned error can't find the container with id c2d99c5a3c0befd38a1cddab6b792b52452d248dc0fc0797971657fbffdd69e0
Jul 28 10:59:14.903194 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.903159    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:14.903311 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.903238    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:14.903311 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:14.903282    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:15.598517 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:15.598487    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerStarted","Data":"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"}
Jul 28 10:59:15.598927 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:15.598535    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerStarted","Data":"c2d99c5a3c0befd38a1cddab6b792b52452d248dc0fc0797971657fbffdd69e0"}
Jul 28 10:59:15.620207 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:15.620159    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podStartSLOduration=1.62014585 podStartE2EDuration="1.62014585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:59:15.61826503 +0000 UTC m=+2180.896202201" watchObservedRunningTime="2026-07-28 10:59:15.62014585 +0000 UTC m=+2180.898083024"
Jul 28 10:59:15.754899 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:15.754864    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:15.760540 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:15.760512    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:16.603854 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:16.603821    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:16.604852 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:16.604831    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:24.545077 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:24.545037    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:59:32.345286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.344061    2544 patch_prober.go:28] interesting pod/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:48184->10.132.0.47:15021: read: connection reset by peer" start-of-body=
Jul 28 10:59:32.345286 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.344123    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:48184->10.132.0.47:15021: read: connection reset by peer"
Jul 28 10:59:32.690712 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.690676    2544 generic.go:356] "Generic (PLEG): container finished" podID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerID="2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb" exitCode=0
Jul 28 10:59:32.690917 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.690747    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerDied","Data":"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"}
Jul 28 10:59:32.691225 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.691206    2544 scope.go:117] "RemoveContainer" containerID="2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"
Jul 28 10:59:32.692509 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.692480    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:32.692619 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.692582    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:32.692678 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:32.692631    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 10:59:33.701980 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:33.701937    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerStarted","Data":"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"}
Jul 28 10:59:33.755403 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:33.755363    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:33.755663 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:33.755641    2544 patch_prober.go:28] interesting pod/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 10:59:33.755754 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:33.755693    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 10:59:34.755976 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:34.755941    2544 patch_prober.go:28] interesting pod/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 10:59:34.756382 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:34.755991    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 10:59:35.755105 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:35.755066    2544 patch_prober.go:28] interesting pod/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 10:59:35.755301 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:35.755126    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 10:59:36.755148 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:36.755111    2544 patch_prober.go:28] interesting pod/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 10:59:36.755538 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:36.755180    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 10:59:37.758702 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:37.758674    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:37.759162 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:37.759024    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:37.759684 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:37.759668    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 10:59:56.242021 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:56.241987    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 10:59:59.299388 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:59.299358    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 10:59:59.299809 ip-10-0-129-31 kubenswrapper[2544]: I0728 10:59:59.299612    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy" containerID="cri-o://a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380" gracePeriod=30
Jul 28 11:00:04.563210 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.563147    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 11:00:04.690854 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.690811    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691035 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.690867    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691035 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.690893    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691035 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.690992    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbfll\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691211 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691043    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691211 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691089    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691211 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691131    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691211 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691153    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691417 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691221    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket\") pod \"d4138de3-5b47-4965-95b7-1957b019bdbc\" (UID: \"d4138de3-5b47-4965-95b7-1957b019bdbc\") "
Jul 28 11:00:04.691417 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691254    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:00:04.691622 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691600    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4138de3-5b47-4965-95b7-1957b019bdbc-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.691841 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691799    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:00:04.691963 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.691883    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:00:04.692159 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.692132    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:00:04.692443 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.692399    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:00:04.693688 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.693664    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll" (OuterVolumeSpecName: "kube-api-access-gbfll") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "kube-api-access-gbfll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:00:04.693688 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.693672    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:00:04.694358 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.694338    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:00:04.694467 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.694451    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4138de3-5b47-4965-95b7-1957b019bdbc" (UID: "d4138de3-5b47-4965-95b7-1957b019bdbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:00:04.792793 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792767    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792793 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792793    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792801    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792810    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792820    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792828    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4138de3-5b47-4965-95b7-1957b019bdbc-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792836    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbfll\" (UniqueName: \"kubernetes.io/projected/d4138de3-5b47-4965-95b7-1957b019bdbc-kube-api-access-gbfll\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.792949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.792844    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4138de3-5b47-4965-95b7-1957b019bdbc-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:00:04.858030 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.857995    2544 generic.go:356] "Generic (PLEG): container finished" podID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerID="a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380" exitCode=0
Jul 28 11:00:04.858030 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.858032    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerDied","Data":"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"}
Jul 28 11:00:04.858287 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.858053    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"
Jul 28 11:00:04.858287 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.858074    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v" event={"ID":"d4138de3-5b47-4965-95b7-1957b019bdbc","Type":"ContainerDied","Data":"c2d99c5a3c0befd38a1cddab6b792b52452d248dc0fc0797971657fbffdd69e0"}
Jul 28 11:00:04.858287 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.858095    2544 scope.go:117] "RemoveContainer" containerID="a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"
Jul 28 11:00:04.867998 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.867981    2544 scope.go:117] "RemoveContainer" containerID="2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"
Jul 28 11:00:04.876160 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.876143    2544 scope.go:117] "RemoveContainer" containerID="a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"
Jul 28 11:00:04.876413 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:00:04.876392    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380\": container with ID starting with a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380 not found: ID does not exist" containerID="a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"
Jul 28 11:00:04.876471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.876421    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380"} err="failed to get container status \"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380\": rpc error: code = NotFound desc = could not find container \"a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380\": container with ID starting with a6532bb15b75f41a4b9a9bc66c74ef00787e77f077f0d163af1b046d601df380 not found: ID does not exist"
Jul 28 11:00:04.876471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.876439    2544 scope.go:117] "RemoveContainer" containerID="2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"
Jul 28 11:00:04.876714 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:00:04.876694    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb\": container with ID starting with 2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb not found: ID does not exist" containerID="2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"
Jul 28 11:00:04.876766 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.876721    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb"} err="failed to get container status \"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb\": rpc error: code = NotFound desc = could not find container \"2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb\": container with ID starting with 2a8e659825686eebbdbd92f274471a8a9a61f3c7529bdd164fc2e472acbd0ebb not found: ID does not exist"
Jul 28 11:00:04.880425 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.880402    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 11:00:04.885240 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:04.885222    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4217be61-openshift-default-7b9c68fb88-8v57v"]
Jul 28 11:00:05.275599 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:05.275476    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" path="/var/lib/kubelet/pods/d4138de3-5b47-4965-95b7-1957b019bdbc/volumes"
Jul 28 11:00:09.141340 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:09.141299    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:00:40.543028 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:00:40.542945    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:02:08.744752 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:08.744671    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:02:20.948386 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:20.948351    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:02:30.755940 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:30.755903    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:02:41.943891 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:41.943852    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:02:55.361980 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:55.361949    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:02:55.365697 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:02:55.365677    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:03:00.141935 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:03:00.141903    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:03:39.665550 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:03:39.665467    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:03:51.848823 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:03:51.848791    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:04:04.451864 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:04.451829    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:04:11.436676 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.436646    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"]
Jul 28 11:04:11.437056 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437041    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 11:04:11.437056 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437053    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 11:04:11.437135 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437072    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:04:11.437135 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437077    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:04:11.437135 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437087    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:04:11.437135 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437092    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:04:11.437266 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437159    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:04:11.437266 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.437172    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="e748513b-e8a8-4734-b419-3a4bf1e3a0d3" containerName="istio-proxy"
Jul 28 11:04:11.440095 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.440077    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:11.442628 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.442606    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:04:11.442745 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.442642    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:04:11.448677 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.448552    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"]
Jul 28 11:04:11.584339 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.584317    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq\" (UID: \"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:11.685284 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.685246    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq\" (UID: \"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:11.685899 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.685882    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq\" (UID: \"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:11.752316 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.752264    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:11.885949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.885923    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"]
Jul 28 11:04:11.888349 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:04:11.888328    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb882ce_77df_4d00_bc2e_7dd6ccde12ca.slice/crio-506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf WatchSource:0}: Error finding container 506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf: Status 404 returned error can't find the container with id 506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf
Jul 28 11:04:11.890291 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:11.890275    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:04:12.082214 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:12.082136    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq" event={"ID":"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca","Type":"ContainerStarted","Data":"506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf"}
Jul 28 11:04:15.101661 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:15.101632    2544 generic.go:356] "Generic (PLEG): container finished" podID="0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" containerID="c982517debd0e6b5e6d9d8f917fb96c179c5f58df1ecfa6ba335cabcc8978195" exitCode=0
Jul 28 11:04:15.102066 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:15.101674    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq" event={"ID":"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca","Type":"ContainerDied","Data":"c982517debd0e6b5e6d9d8f917fb96c179c5f58df1ecfa6ba335cabcc8978195"}
Jul 28 11:04:16.246394 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:16.246369    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:16.327216 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:16.327189    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca\") pod \"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca\" (UID: \"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca\") "
Jul 28 11:04:16.327545 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:16.327525    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" (UID: "0eb882ce-77df-4d00-bc2e-7dd6ccde12ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:04:16.428462 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:16.428384    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:04:17.113908 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:17.113880    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"
Jul 28 11:04:17.114069 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:17.113875    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq" event={"ID":"0eb882ce-77df-4d00-bc2e-7dd6ccde12ca","Type":"ContainerDied","Data":"506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf"}
Jul 28 11:04:17.114069 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:17.114010    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="506decb626282fa8af73ae63a91a44ca006f6ea8205d1ccd2098f373b35b4bcf"
Jul 28 11:04:25.350638 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:25.350602    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:04:31.511440 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:31.511401    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"]
Jul 28 11:04:31.517565 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:31.517529    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0b092d88-xjzzq"]
Jul 28 11:04:33.269708 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:33.269676    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" path="/var/lib/kubelet/pods/0eb882ce-77df-4d00-bc2e-7dd6ccde12ca/volumes"
Jul 28 11:04:35.942303 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:04:35.942272    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:05:33.466218 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:05:33.466182    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:05:41.649544 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:05:41.649504    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:06:25.043705 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:25.043613    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:06:27.198614 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.198577    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:06:27.199222 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.199199    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" containerName="revoke-keys"
Jul 28 11:06:27.199288 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.199227    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" containerName="revoke-keys"
Jul 28 11:06:27.199360 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.199346    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4138de3-5b47-4965-95b7-1957b019bdbc" containerName="istio-proxy"
Jul 28 11:06:27.199396 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.199370    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eb882ce-77df-4d00-bc2e-7dd6ccde12ca" containerName="revoke-keys"
Jul 28 11:06:27.202994 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.202973    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.205417 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.205398    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2298f8-openshift-default-dockercfg-67tqs\""
Jul 28 11:06:27.213275 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.213254    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:06:27.259208 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259182    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259218    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtkxv\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259247    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259302    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259437 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259369    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259437 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259387    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259437 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259416    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259437 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259431    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.259587 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.259446    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.359997 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.359958    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtkxv\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.359997 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360003    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360030    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360073    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360089    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360125    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360147    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360171    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360244 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360206    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360680 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360648    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360811 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360649    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.360811 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360769    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.361099 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.360901    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.361099 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.361066    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.362466 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.362447    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.362908 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.362886    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.367519 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.367499    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.368004 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.367980    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtkxv\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv\") pod \"e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") " pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.514255 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.514187    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:27.660157 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.660128    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:06:27.661176 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:06:27.661151    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95552ff_6ede_4226_bae8_90946af19151.slice/crio-09bbddf517a55b099cbe818124f19efe55115062fd6a30ed50086c375db31238 WatchSource:0}: Error finding container 09bbddf517a55b099cbe818124f19efe55115062fd6a30ed50086c375db31238: Status 404 returned error can't find the container with id 09bbddf517a55b099cbe818124f19efe55115062fd6a30ed50086c375db31238
Jul 28 11:06:27.663749 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.663720    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:27.663819 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.663782    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:27.663819 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.663813    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:27.761593 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.761539    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerStarted","Data":"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"}
Jul 28 11:06:27.761838 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.761810    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerStarted","Data":"09bbddf517a55b099cbe818124f19efe55115062fd6a30ed50086c375db31238"}
Jul 28 11:06:27.781326 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:27.781216    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podStartSLOduration=0.781199699 podStartE2EDuration="781.199699ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:06:27.779495338 +0000 UTC m=+2613.057432513" watchObservedRunningTime="2026-07-28 11:06:27.781199699 +0000 UTC m=+2613.059136872"
Jul 28 11:06:28.515112 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:28.515075    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:28.520711 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:28.520682    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:28.767390 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:28.767320    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:28.768295 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:28.768274    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:40.839216 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.839136    2544 generic.go:356] "Generic (PLEG): container finished" podID="f95552ff-6ede-4226-bae8-90946af19151" containerID="0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43" exitCode=0
Jul 28 11:06:40.839635 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.839215    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerDied","Data":"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"}
Jul 28 11:06:40.839635 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.839616    2544 scope.go:117] "RemoveContainer" containerID="0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"
Jul 28 11:06:40.840749 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.840716    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:40.840819 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.840787    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:40.840873 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:40.840819    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:06:41.137500 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:41.137465    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:06:41.846458 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:41.846430    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerStarted","Data":"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"}
Jul 28 11:06:42.515675 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:42.515634    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:42.516129 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:42.516100    2544 patch_prober.go:28] interesting pod/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 11:06:42.516267 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:42.516147    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 11:06:43.515049 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:43.514956    2544 patch_prober.go:28] interesting pod/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 11:06:43.515049 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:43.515025    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 11:06:43.767608 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:43.767493    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:44.515253 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:44.515214    2544 patch_prober.go:28] interesting pod/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 11:06:44.515674 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:44.515274    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 11:06:45.514833 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:45.514789    2544 patch_prober.go:28] interesting pod/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 11:06:45.515051 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:45.514862    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 11:06:46.518921 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:46.518889    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:46.519868 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:46.519845    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:06:59.458170 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:06:59.458129    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:07:13.444010 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:13.443967    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:07:35.553932 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.553894    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:07:35.557334 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.557316    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.561047 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.561025    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2298f8\"/\"default-dockercfg-zg7dd\""
Jul 28 11:07:35.561166 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.561054    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2298f8\"/\"test-model-2298f8-kserve-self-signed-certs\""
Jul 28 11:07:35.561166 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.561091    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2298f8\"/\"kube-root-ca.crt\""
Jul 28 11:07:35.561291 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.561177    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2298f8\"/\"openshift-service-ca.crt\""
Jul 28 11:07:35.567697 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.567676    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:07:35.665871 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.665840    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thhsm\" (UniqueName: \"kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.666011 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.665881    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.666011 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.665920    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.666011 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.665956    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.666149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.666014    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.666149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.666034    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767325 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767293    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thhsm\" (UniqueName: \"kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767338    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767375    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767397    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767451    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767472    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767860 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767840    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767945 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767874    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.767945 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.767921    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.769807 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.769787    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.770050 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.770034    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.776284 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.776260    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thhsm\" (UniqueName: \"kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm\") pod \"test-model-2298f8-kserve-7fb7d7c6cf-smmb8\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") " pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:35.868180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:35.868149    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:36.019170 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:36.019146    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:07:36.020953 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:07:36.020926    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a88b815_5233_4241_8e41_912d9446b95e.slice/crio-33251008d4afa7b8721f15d4f87875d1b7568bd09ee7ad812c7650f76eeea093 WatchSource:0}: Error finding container 33251008d4afa7b8721f15d4f87875d1b7568bd09ee7ad812c7650f76eeea093: Status 404 returned error can't find the container with id 33251008d4afa7b8721f15d4f87875d1b7568bd09ee7ad812c7650f76eeea093
Jul 28 11:07:36.122672 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:36.122587    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" event={"ID":"1a88b815-5233-4241-8e41-912d9446b95e","Type":"ContainerStarted","Data":"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"}
Jul 28 11:07:36.122672 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:36.122642    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" event={"ID":"1a88b815-5233-4241-8e41-912d9446b95e","Type":"ContainerStarted","Data":"33251008d4afa7b8721f15d4f87875d1b7568bd09ee7ad812c7650f76eeea093"}
Jul 28 11:07:36.140703 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:36.140659    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" podStartSLOduration=1.1406447 podStartE2EDuration="1.1406447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:07:36.139279599 +0000 UTC m=+2681.417216773" watchObservedRunningTime="2026-07-28 11:07:36.1406447 +0000 UTC m=+2681.418581874"
Jul 28 11:07:41.142009 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:41.141976    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:07:45.869343 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:45.869303    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:45.869940 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:45.869359    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:45.881885 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:45.881862    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:46.183357 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:46.183276    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:07:55.392587 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:55.392494    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:07:55.398450 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:07:55.398428    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:08:16.070885 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.070855    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:08:16.223234 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.223200    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"]
Jul 28 11:08:16.226809 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.226787    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:16.229249 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.229229    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:08:16.229358 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.229288    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:08:16.232529 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.232507    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"]
Jul 28 11:08:16.328328 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.328242    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-730a1a-lt749\" (UID: \"51ef074a-fdc1-4e56-8478-46ae794bd8cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:16.429237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.429212    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-730a1a-lt749\" (UID: \"51ef074a-fdc1-4e56-8478-46ae794bd8cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:16.429809 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.429791    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-730a1a-lt749\" (UID: \"51ef074a-fdc1-4e56-8478-46ae794bd8cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:16.538781 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.538755    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:16.881965 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:16.881871    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"]
Jul 28 11:08:16.884580 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:08:16.884529    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ef074a_fdc1_4e56_8478_46ae794bd8cc.slice/crio-b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251 WatchSource:0}: Error finding container b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251: Status 404 returned error can't find the container with id b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251
Jul 28 11:08:17.338074 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:17.338040    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749" event={"ID":"51ef074a-fdc1-4e56-8478-46ae794bd8cc","Type":"ContainerStarted","Data":"5b01c58cb3a56691962473878888d54dc494b882e197362175863145163e2602"}
Jul 28 11:08:17.338491 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:17.338091    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749" event={"ID":"51ef074a-fdc1-4e56-8478-46ae794bd8cc","Type":"ContainerStarted","Data":"b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251"}
Jul 28 11:08:17.353469 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:17.353416    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749" podStartSLOduration=1.353402488 podStartE2EDuration="1.353402488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:08:17.352884129 +0000 UTC m=+2722.630821304" watchObservedRunningTime="2026-07-28 11:08:17.353402488 +0000 UTC m=+2722.631339661"
Jul 28 11:08:18.345644 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:18.345607    2544 generic.go:356] "Generic (PLEG): container finished" podID="51ef074a-fdc1-4e56-8478-46ae794bd8cc" containerID="5b01c58cb3a56691962473878888d54dc494b882e197362175863145163e2602" exitCode=0
Jul 28 11:08:18.346055 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:18.345699    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749" event={"ID":"51ef074a-fdc1-4e56-8478-46ae794bd8cc","Type":"ContainerDied","Data":"5b01c58cb3a56691962473878888d54dc494b882e197362175863145163e2602"}
Jul 28 11:08:19.502192 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:19.502170    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:19.560772 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:19.560749    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca\") pod \"51ef074a-fdc1-4e56-8478-46ae794bd8cc\" (UID: \"51ef074a-fdc1-4e56-8478-46ae794bd8cc\") "
Jul 28 11:08:19.561114 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:19.561093    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51ef074a-fdc1-4e56-8478-46ae794bd8cc" (UID: "51ef074a-fdc1-4e56-8478-46ae794bd8cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:08:19.661872 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:19.661800    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ef074a-fdc1-4e56-8478-46ae794bd8cc-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:20.357066 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:20.357035    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749" event={"ID":"51ef074a-fdc1-4e56-8478-46ae794bd8cc","Type":"ContainerDied","Data":"b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251"}
Jul 28 11:08:20.357066 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:20.357058    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"
Jul 28 11:08:20.357285 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:20.357089    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b076322e3dc0050b6b91959d7d72a04a267799833771443c185761aeb86cd251"
Jul 28 11:08:22.552201 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:22.552163    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:08:31.091300 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.091256    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" podUID="1a88b815-5233-4241-8e41-912d9446b95e" containerName="main" containerID="cri-o://2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294" gracePeriod=45
Jul 28 11:08:31.310914 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.310883    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"]
Jul 28 11:08:31.313316 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.313291    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-730a1a-lt749"]
Jul 28 11:08:31.348643 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.348594    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:08:31.414597 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.414539    2544 generic.go:356] "Generic (PLEG): container finished" podID="1a88b815-5233-4241-8e41-912d9446b95e" containerID="2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294" exitCode=0
Jul 28 11:08:31.414746 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.414601    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" event={"ID":"1a88b815-5233-4241-8e41-912d9446b95e","Type":"ContainerDied","Data":"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"}
Jul 28 11:08:31.414746 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.414645    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8" event={"ID":"1a88b815-5233-4241-8e41-912d9446b95e","Type":"ContainerDied","Data":"33251008d4afa7b8721f15d4f87875d1b7568bd09ee7ad812c7650f76eeea093"}
Jul 28 11:08:31.414746 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.414664    2544 scope.go:117] "RemoveContainer" containerID="2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"
Jul 28 11:08:31.414746 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.414613    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"
Jul 28 11:08:31.423522 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.423502    2544 scope.go:117] "RemoveContainer" containerID="2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"
Jul 28 11:08:31.423812 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:08:31.423795    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294\": container with ID starting with 2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294 not found: ID does not exist" containerID="2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"
Jul 28 11:08:31.423862 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.423822    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294"} err="failed to get container status \"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294\": rpc error: code = NotFound desc = could not find container \"2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294\": container with ID starting with 2d6ef732c64e17f75f392b982bb7751d87bb5e8dfa5fbeb53c43ef8f14a21294 not found: ID does not exist"
Jul 28 11:08:31.468671 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468648    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.468787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468684    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.468787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468713    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.468787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468743    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thhsm\" (UniqueName: \"kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.468938 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468830    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.468938 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468854    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home\") pod \"1a88b815-5233-4241-8e41-912d9446b95e\" (UID: \"1a88b815-5233-4241-8e41-912d9446b95e\") "
Jul 28 11:08:31.469047 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.468985    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:08:31.469105 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.469087    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:08:31.469173 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.469152    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home" (OuterVolumeSpecName: "home") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:08:31.469250 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.469236    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-home\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.469305 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.469253    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-model-cache\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.469305 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.469264    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-tmp-dir\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.471107 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.471079    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm" (OuterVolumeSpecName: "dshm") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:08:31.471199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.471129    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:08:31.471199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.471137    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm" (OuterVolumeSpecName: "kube-api-access-thhsm") pod "1a88b815-5233-4241-8e41-912d9446b95e" (UID: "1a88b815-5233-4241-8e41-912d9446b95e"). InnerVolumeSpecName "kube-api-access-thhsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:08:31.570466 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.570437    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a88b815-5233-4241-8e41-912d9446b95e-dshm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.570466 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.570463    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a88b815-5233-4241-8e41-912d9446b95e-tls-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.570466 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.570474    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thhsm\" (UniqueName: \"kubernetes.io/projected/1a88b815-5233-4241-8e41-912d9446b95e-kube-api-access-thhsm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:31.735928 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.735901    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:08:31.739847 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:31.739824    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2298f8/test-model-2298f8-kserve-7fb7d7c6cf-smmb8"]
Jul 28 11:08:32.054573 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:32.054486    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:08:33.270367 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:33.270335    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a88b815-5233-4241-8e41-912d9446b95e" path="/var/lib/kubelet/pods/1a88b815-5233-4241-8e41-912d9446b95e/volumes"
Jul 28 11:08:33.270878 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:33.270855    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ef074a-fdc1-4e56-8478-46ae794bd8cc" path="/var/lib/kubelet/pods/51ef074a-fdc1-4e56-8478-46ae794bd8cc/volumes"
Jul 28 11:08:38.202461 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.202421    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"]
Jul 28 11:08:38.203012 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.202992    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a88b815-5233-4241-8e41-912d9446b95e" containerName="main"
Jul 28 11:08:38.203098 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.203016    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a88b815-5233-4241-8e41-912d9446b95e" containerName="main"
Jul 28 11:08:38.203098 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.203035    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ef074a-fdc1-4e56-8478-46ae794bd8cc" containerName="revoke-keys"
Jul 28 11:08:38.203098 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.203043    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ef074a-fdc1-4e56-8478-46ae794bd8cc" containerName="revoke-keys"
Jul 28 11:08:38.203242 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.203164    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a88b815-5233-4241-8e41-912d9446b95e" containerName="main"
Jul 28 11:08:38.203242 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.203177    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ef074a-fdc1-4e56-8478-46ae794bd8cc" containerName="revoke-keys"
Jul 28 11:08:38.206471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.206448    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:38.208787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.208770    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:08:38.208894 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.208771    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:08:38.213146 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.213124    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"]
Jul 28 11:08:38.328412 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.328378    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr\" (UID: \"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:38.429516 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.429488    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr\" (UID: \"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:38.430076 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.430058    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr\" (UID: \"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:38.518113 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.518006    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:38.659023 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:38.658997    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"]
Jul 28 11:08:38.661202 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:08:38.661177    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79dd5d0_31a7_4928_b1a1_bf3ad1aaaa3b.slice/crio-2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6 WatchSource:0}: Error finding container 2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6: Status 404 returned error can't find the container with id 2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6
Jul 28 11:08:39.464845 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:39.464813    2544 generic.go:356] "Generic (PLEG): container finished" podID="f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" containerID="732e8e981d6929d5da33653b4761a0ff220696da7078540e6b2dae5c1f16fcd2" exitCode=0
Jul 28 11:08:39.465205 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:39.464902    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr" event={"ID":"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b","Type":"ContainerDied","Data":"732e8e981d6929d5da33653b4761a0ff220696da7078540e6b2dae5c1f16fcd2"}
Jul 28 11:08:39.465205 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:39.464997    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr" event={"ID":"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b","Type":"ContainerStarted","Data":"2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6"}
Jul 28 11:08:40.610439 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:40.610414    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:40.749001 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:40.748936    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca\") pod \"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b\" (UID: \"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b\") "
Jul 28 11:08:40.749265 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:40.749246    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" (UID: "f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:08:40.849737 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:40.849714    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:08:41.475988 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:41.475959    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"
Jul 28 11:08:41.475988 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:41.475978    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr" event={"ID":"f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b","Type":"ContainerDied","Data":"2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6"}
Jul 28 11:08:41.476185 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:41.476020    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bd13d304c60059d883a9abc92a407d79aeed159a47aab51510366bbd9daecf6"
Jul 28 11:08:51.939126 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:51.939091    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:08:53.273882 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:53.273844    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"]
Jul 28 11:08:53.278587 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:53.277404    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2298f8-vdrcr"]
Jul 28 11:08:55.270384 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:55.270355    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" path="/var/lib/kubelet/pods/f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b/volumes"
Jul 28 11:08:59.772061 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:59.772027    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:08:59.772519 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:08:59.772233    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" containerID="cri-o://c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7" gracePeriod=30
Jul 28 11:09:01.521498 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:01.521455    2544 patch_prober.go:28] interesting pod/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 11:09:01.521903 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:01.521513    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 11:09:05.043971 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.043917    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:09:05.155690 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155663    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155690 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155692    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155715    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155742    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155777    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155811    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtkxv\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155834    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.155923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155914    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.156237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155966    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data\") pod \"f95552ff-6ede-4226-bae8-90946af19151\" (UID: \"f95552ff-6ede-4226-bae8-90946af19151\") "
Jul 28 11:09:05.156237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155960    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:09:05.156237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156011    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:09:05.156237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.155984    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:09:05.156237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156134    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:09:05.156487 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156430    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.156487 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156451    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.156487 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156466    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95552ff-6ede-4226-bae8-90946af19151-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.156487 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.156481    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.157063 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.157038    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data" (OuterVolumeSpecName: "istio-data") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:09:05.158203 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.158183    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:09:05.158203 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.158187    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv" (OuterVolumeSpecName: "kube-api-access-vtkxv") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "kube-api-access-vtkxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:09:05.158621 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.158604    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:09:05.158701 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.158625    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token" (OuterVolumeSpecName: "istio-token") pod "f95552ff-6ede-4226-bae8-90946af19151" (UID: "f95552ff-6ede-4226-bae8-90946af19151"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:09:05.257183 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.257156    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95552ff-6ede-4226-bae8-90946af19151-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.257183 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.257179    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.257183 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.257188    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.257363 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.257196    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtkxv\" (UniqueName: \"kubernetes.io/projected/f95552ff-6ede-4226-bae8-90946af19151-kube-api-access-vtkxv\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.257363 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.257205    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95552ff-6ede-4226-bae8-90946af19151-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:05.597279 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.597245    2544 generic.go:356] "Generic (PLEG): container finished" podID="f95552ff-6ede-4226-bae8-90946af19151" containerID="c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7" exitCode=0
Jul 28 11:09:05.597461 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.597301    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"
Jul 28 11:09:05.597461 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.597337    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerDied","Data":"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"}
Jul 28 11:09:05.597461 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.597426    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz" event={"ID":"f95552ff-6ede-4226-bae8-90946af19151","Type":"ContainerDied","Data":"09bbddf517a55b099cbe818124f19efe55115062fd6a30ed50086c375db31238"}
Jul 28 11:09:05.597461 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.597455    2544 scope.go:117] "RemoveContainer" containerID="c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"
Jul 28 11:09:05.606671 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.606654    2544 scope.go:117] "RemoveContainer" containerID="0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"
Jul 28 11:09:05.617698 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.617675    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:09:05.619495 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.619478    2544 scope.go:117] "RemoveContainer" containerID="c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"
Jul 28 11:09:05.619842 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:09:05.619810    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7\": container with ID starting with c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7 not found: ID does not exist" containerID="c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"
Jul 28 11:09:05.619942 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.619846    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7"} err="failed to get container status \"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7\": rpc error: code = NotFound desc = could not find container \"c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7\": container with ID starting with c780645cf2975d4b205c3b91274b96ef0cbc68358624e6d0a70644917ff339b7 not found: ID does not exist"
Jul 28 11:09:05.619942 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.619862    2544 scope.go:117] "RemoveContainer" containerID="0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"
Jul 28 11:09:05.620165 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:09:05.620146    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43\": container with ID starting with 0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43 not found: ID does not exist" containerID="0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"
Jul 28 11:09:05.620214 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.620176    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43"} err="failed to get container status \"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43\": rpc error: code = NotFound desc = could not find container \"0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43\": container with ID starting with 0a647bac67ae3a8eff03f542182ad22f90f30545216df08556b538753157ef43 not found: ID does not exist"
Jul 28 11:09:05.621186 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:05.621169    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2298f8-openshift-default-7f6b4fd7d6-gmcdz"]
Jul 28 11:09:07.271951 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:07.271920    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f95552ff-6ede-4226-bae8-90946af19151" path="/var/lib/kubelet/pods/f95552ff-6ede-4226-bae8-90946af19151/volumes"
Jul 28 11:09:08.543858 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:08.543829    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:09:09.201706 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.201673    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:09:09.202099 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202078    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" containerName="revoke-keys"
Jul 28 11:09:09.202099 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202099    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" containerName="revoke-keys"
Jul 28 11:09:09.202299 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202119    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.202299 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202128    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.202299 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202155    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.202299 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202164    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.202504 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202349    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79dd5d0-31a7-4928-b1a1-bf3ad1aaaa3b" containerName="revoke-keys"
Jul 28 11:09:09.202504 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202373    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.202504 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.202386    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95552ff-6ede-4226-bae8-90946af19151" containerName="istio-proxy"
Jul 28 11:09:09.207193 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.207175    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.209462 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.209441    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-86170f-openshift-default-dockercfg-sskvj\""
Jul 28 11:09:09.218481 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.218460    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:09:09.291978 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.291921    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292128 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292021    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292128 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292116    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292265 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292170    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292265 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292219    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292374 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292292    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292374 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292338    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292374 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292364    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.292594 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.292437    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsbpt\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.393821 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.393789    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394116 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394089    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394304 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394286    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394541 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394379    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394755 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394733    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsbpt\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394983 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394820    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.394983 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394948    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395145 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.394988    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395145 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395036    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395145 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395061    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395145 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395119    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395399 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395372    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395489 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395448    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.395692 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.395645    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.398231 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.398156    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.398512 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.398489    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.403299 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.403280    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.403549 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.403531    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsbpt\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt\") pod \"e2e-api-b-86170f-openshift-default-775775dbd6-shbjv\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") " pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.518913 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.518838    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:09.662346 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.662322    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:09:09.663978 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:09:09.663953    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9e20fa_4451_4e7d_a4f3_9476509f3538.slice/crio-e7a17df643402207f9bafc4a6612e82f133d1fc49672c3ce6a69575713f40d7c WatchSource:0}: Error finding container e7a17df643402207f9bafc4a6612e82f133d1fc49672c3ce6a69575713f40d7c: Status 404 returned error can't find the container with id e7a17df643402207f9bafc4a6612e82f133d1fc49672c3ce6a69575713f40d7c
Jul 28 11:09:09.666042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.666013    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:09.666145 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.666087    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:09.666277 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:09.666155    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:10.633253 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:10.633220    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerStarted","Data":"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"}
Jul 28 11:09:10.633427 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:10.633295    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerStarted","Data":"e7a17df643402207f9bafc4a6612e82f133d1fc49672c3ce6a69575713f40d7c"}
Jul 28 11:09:10.653281 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:10.653213    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podStartSLOduration=1.6531996420000001 podStartE2EDuration="1.653199642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:09:10.652748299 +0000 UTC m=+2775.930685479" watchObservedRunningTime="2026-07-28 11:09:10.653199642 +0000 UTC m=+2775.931136816"
Jul 28 11:09:11.519694 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:11.519665    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:11.524696 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:11.524671    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:11.638783 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:11.638753    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:11.639703 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:11.639683    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:20.343199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:20.343117    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:09:27.679734 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:27.679690    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 11:09:27.680167 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:27.679777    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 11:09:29.742097 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.742061    2544 generic.go:356] "Generic (PLEG): container finished" podID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerID="328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12" exitCode=0
Jul 28 11:09:29.742515 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.742138    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerDied","Data":"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"}
Jul 28 11:09:29.742602 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.742586    2544 scope.go:117] "RemoveContainer" containerID="328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"
Jul 28 11:09:29.743677 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.743653    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:29.743753 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.743717    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:29.743753 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:29.743749    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:09:30.748779 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:30.748747    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerStarted","Data":"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"}
Jul 28 11:09:31.519983 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:31.519945    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:31.520193 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:31.520174    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 11:09:31.520239 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:31.520217    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 11:09:32.520038 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:32.519998    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 11:09:32.520442 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:32.520089    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 11:09:33.519931 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:33.519894    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 11:09:33.520125 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:33.519977    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 11:09:34.519987 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:34.519943    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 11:09:34.520192 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:34.520012    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 11:09:35.523235 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:35.523201    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:35.523682 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:35.523594    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:35.524331 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:35.524316    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:09:43.243816 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:43.243784    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:09:52.135295 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.135262    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"]
Jul 28 11:09:52.137830 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.137814    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:52.141389 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.141362    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:09:52.141389 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.141395    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:09:52.146871 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.146846    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"]
Jul 28 11:09:52.160347 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.160306    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86170f-lrshp\" (UID: \"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:52.260936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.260896    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86170f-lrshp\" (UID: \"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:52.261548 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.261522    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-86170f-lrshp\" (UID: \"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:52.453073 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.452985    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:52.591927 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.591903    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"]
Jul 28 11:09:52.594948 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:09:52.594918    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ecb6b4_2e87_4bd9_b8e0_cb2d9d95be3e.slice/crio-f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e WatchSource:0}: Error finding container f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e: Status 404 returned error can't find the container with id f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e
Jul 28 11:09:52.869005 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:52.868969    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp" event={"ID":"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e","Type":"ContainerStarted","Data":"f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e"}
Jul 28 11:09:53.876618 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:53.876585    2544 generic.go:356] "Generic (PLEG): container finished" podID="a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" containerID="055eae97a257f311afcf28ca32b43c771dc8b4c056ea67d983a1cce6022dc428" exitCode=0
Jul 28 11:09:53.876999 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:53.876620    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp" event={"ID":"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e","Type":"ContainerDied","Data":"055eae97a257f311afcf28ca32b43c771dc8b4c056ea67d983a1cce6022dc428"}
Jul 28 11:09:55.022623 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.022597    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:55.083064 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.083034    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca\") pod \"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e\" (UID: \"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e\") "
Jul 28 11:09:55.083350 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.083330    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" (UID: "a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:09:55.184178 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.184106    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:09:55.887700 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.887677    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"
Jul 28 11:09:55.887918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.887676    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp" event={"ID":"a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e","Type":"ContainerDied","Data":"f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e"}
Jul 28 11:09:55.887918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:09:55.887787    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4ec08bfdd411fb4377ddf8abd35b2b7b0a19cab111802617bead447f0d0b47e"
Jul 28 11:10:12.200574 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:12.200516    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"]
Jul 28 11:10:12.202603 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:12.202577    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-86170f-lrshp"]
Jul 28 11:10:12.342217 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:12.342188    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:10:13.269991 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:13.269958    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" path="/var/lib/kubelet/pods/a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e/volumes"
Jul 28 11:10:18.716701 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:18.716617    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:10:18.717292 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:18.716954    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" containerID="cri-o://19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd" gracePeriod=30
Jul 28 11:10:20.525726 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:20.525693    2544 patch_prober.go:28] interesting pod/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 11:10:20.526102 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:20.525752    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 11:10:24.002781 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.002753    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:10:24.037511 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.037480    2544 generic.go:356] "Generic (PLEG): container finished" podID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerID="19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd" exitCode=0
Jul 28 11:10:24.037676 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.037524    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"
Jul 28 11:10:24.037676 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.037587    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerDied","Data":"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"}
Jul 28 11:10:24.037676 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.037649    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv" event={"ID":"2a9e20fa-4451-4e7d-a4f3-9476509f3538","Type":"ContainerDied","Data":"e7a17df643402207f9bafc4a6612e82f133d1fc49672c3ce6a69575713f40d7c"}
Jul 28 11:10:24.037793 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.037682    2544 scope.go:117] "RemoveContainer" containerID="19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"
Jul 28 11:10:24.046804 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.046785    2544 scope.go:117] "RemoveContainer" containerID="328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"
Jul 28 11:10:24.054915 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.054887    2544 scope.go:117] "RemoveContainer" containerID="19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"
Jul 28 11:10:24.055182 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:10:24.055165    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd\": container with ID starting with 19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd not found: ID does not exist" containerID="19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"
Jul 28 11:10:24.055232 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.055193    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd"} err="failed to get container status \"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd\": rpc error: code = NotFound desc = could not find container \"19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd\": container with ID starting with 19783463900c34cdc4a0a0555d4bf7de517b3750cd92f48045486c9f3c4b7ddd not found: ID does not exist"
Jul 28 11:10:24.055232 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.055209    2544 scope.go:117] "RemoveContainer" containerID="328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"
Jul 28 11:10:24.055445 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:10:24.055427    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12\": container with ID starting with 328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12 not found: ID does not exist" containerID="328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"
Jul 28 11:10:24.055505 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.055448    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12"} err="failed to get container status \"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12\": rpc error: code = NotFound desc = could not find container \"328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12\": container with ID starting with 328349482f994d15c4c34435946b80157208893efd9ebcff3ca4c679f46b6b12 not found: ID does not exist"
Jul 28 11:10:24.159166 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159134    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159433 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159192    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159433 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159225    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159433 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159400    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:10:24.159433 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159405    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159469    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159500    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159516    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159540    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159604    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsbpt\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt\") pod \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\" (UID: \"2a9e20fa-4451-4e7d-a4f3-9476509f3538\") "
Jul 28 11:10:24.159950 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159732    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:10:24.159950 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.159849    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:10:24.160052 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160018    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:10:24.160160 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160137    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:10:24.160279 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160265    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.160341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160286    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.160341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160299    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.160341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160312    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.160341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.160326    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.161581 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.161524    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:10:24.161876 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.161854    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt" (OuterVolumeSpecName: "kube-api-access-lsbpt") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "kube-api-access-lsbpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:10:24.162154 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.162124    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:10:24.162154 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.162135    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a9e20fa-4451-4e7d-a4f3-9476509f3538" (UID: "2a9e20fa-4451-4e7d-a4f3-9476509f3538"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:10:24.261840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.261764    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.261840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.261788    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.261840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.261797    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.261840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.261806    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsbpt\" (UniqueName: \"kubernetes.io/projected/2a9e20fa-4451-4e7d-a4f3-9476509f3538-kube-api-access-lsbpt\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:10:24.360062 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.360033    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:10:24.365642 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:24.365618    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-86170f-openshift-default-775775dbd6-shbjv"]
Jul 28 11:10:25.270999 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:25.270964    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" path="/var/lib/kubelet/pods/2a9e20fa-4451-4e7d-a4f3-9476509f3538/volumes"
Jul 28 11:10:27.404167 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404132    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:10:27.404691 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404638    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" containerName="revoke-keys"
Jul 28 11:10:27.404691 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404656    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" containerName="revoke-keys"
Jul 28 11:10:27.404814 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404694    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.404814 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404704    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.404814 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404723    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.404814 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404732    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.405007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404845    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ecb6b4-2e87-4bd9-b8e0-cb2d9d95be3e" containerName="revoke-keys"
Jul 28 11:10:27.405007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404857    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.405007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.404868    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9e20fa-4451-4e7d-a4f3-9476509f3538" containerName="istio-proxy"
Jul 28 11:10:27.409950 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.409929    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.412606 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.412548    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-683bdc-openshift-default-dockercfg-lr5hj\""
Jul 28 11:10:27.427684 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.427657    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:10:27.591784 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.591743    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqb4p\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.591980 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.591808    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.591980 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.591890    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.591980 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.591969    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.592149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.592013    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.592149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.592075    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.592149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.592104    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.592149 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.592130    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.592321 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.592190    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693609 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693492    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693609 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693546    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqb4p\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693625    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693663    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693722    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693761    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693824    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.693856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693848    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694159 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.693870    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.694315    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694521 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.694492    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694656 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.694392    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694656 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.694540    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.694656 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.694360    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.696473 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.696447    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.696975 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.696960    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.706977 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.706953    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.707062 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.706958    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqb4p\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p\") pod \"e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") " pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.726652 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.726628    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:27.878074 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.878045    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:10:27.879757 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:10:27.879728    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294fbe09_9481_4814_a879_ca4f48058f23.slice/crio-8274e5d3f1e05dd62c9a8bee6792ccb1ae1877dafef6958f1ba90c88f7f6f805 WatchSource:0}: Error finding container 8274e5d3f1e05dd62c9a8bee6792ccb1ae1877dafef6958f1ba90c88f7f6f805: Status 404 returned error can't find the container with id 8274e5d3f1e05dd62c9a8bee6792ccb1ae1877dafef6958f1ba90c88f7f6f805
Jul 28 11:10:27.882064 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.882028    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:27.882133 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.882121    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:27.882181 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:27.882165    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:28.063421 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:28.063381    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerStarted","Data":"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"}
Jul 28 11:10:28.063615 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:28.063446    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerStarted","Data":"8274e5d3f1e05dd62c9a8bee6792ccb1ae1877dafef6958f1ba90c88f7f6f805"}
Jul 28 11:10:28.083335 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:28.083291    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podStartSLOduration=1.083276488 podStartE2EDuration="1.083276488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:10:28.081866363 +0000 UTC m=+2853.359803538" watchObservedRunningTime="2026-07-28 11:10:28.083276488 +0000 UTC m=+2853.361213661"
Jul 28 11:10:28.727657 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:28.727611    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:28.733762 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:28.733732    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:29.069492 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:29.069409    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:29.070290 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:29.070274    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:30.445154 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:30.445117    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:10:42.343730 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:42.343693    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:10:44.386194 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:44.386154    2544 patch_prober.go:28] interesting pod/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:36296->10.132.0.55:15021: read: connection reset by peer" start-of-body=
Jul 28 11:10:44.386651 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:44.386226    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:36296->10.132.0.55:15021: read: connection reset by peer"
Jul 28 11:10:45.162007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.161970    2544 generic.go:356] "Generic (PLEG): container finished" podID="294fbe09-9481-4814-a879-ca4f48058f23" containerID="9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5" exitCode=0
Jul 28 11:10:45.162174 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.162042    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerDied","Data":"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"}
Jul 28 11:10:45.162424 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.162410    2544 scope.go:117] "RemoveContainer" containerID="9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"
Jul 28 11:10:45.163496 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.163450    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:45.163575 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.163531    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:45.163629 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:45.163586    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:10:46.169263 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:46.169228    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerStarted","Data":"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"}
Jul 28 11:10:46.727212 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:46.727169    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:46.727412 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:46.727391    2544 patch_prober.go:28] interesting pod/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 11:10:46.727459 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:46.727441    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 11:10:47.728049 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:47.728013    2544 patch_prober.go:28] interesting pod/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 11:10:47.728446 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:47.728079    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 11:10:48.728032 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:48.727993    2544 patch_prober.go:28] interesting pod/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 11:10:48.728214 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:48.728050    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 11:10:49.727177 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:49.727147    2544 patch_prober.go:28] interesting pod/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 11:10:49.727355 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:49.727209    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 11:10:50.730589 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:50.730542    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:50.731063 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:50.730939    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:50.731472 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:50.731452    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:10:55.239803 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:55.239769    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:10:55.485940 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:10:55.485911    2544 scope.go:117] "RemoveContainer" containerID="c982517debd0e6b5e6d9d8f917fb96c179c5f58df1ecfa6ba335cabcc8978195"
Jul 28 11:11:16.659610 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.659518    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:11:16.663188 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.663165    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.665878 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.665852    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"openshift-service-ca.crt\""
Jul 28 11:11:16.666926 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.666910    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"kube-root-ca.crt\""
Jul 28 11:11:16.667002 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.666950    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"auth-test-model-683bdc-kserve-self-signed-certs\""
Jul 28 11:11:16.667049 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.666957    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"default-dockercfg-k9r75\""
Jul 28 11:11:16.671882 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.671860    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:11:16.852119 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852087    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.852307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852126    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.852307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852151    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.852307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852171    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.852307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852239    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jfsx\" (UniqueName: \"kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.852307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.852306    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953322 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953234    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jfsx\" (UniqueName: \"kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953322 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953279    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953551 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953331    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953551 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953356    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953551 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953378    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953551 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953411    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953871 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953850    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953932 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953892    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.953991 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.953936    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.955797 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.955779    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.956063 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.956047    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.962069 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.962050    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jfsx\" (UniqueName: \"kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx\") pod \"auth-test-model-683bdc-kserve-64d6c65f9f-j855r\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") " pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:16.974109 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:16.974089    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:17.117174 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:17.117150    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:11:17.119623 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:11:17.119593    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4311eeaa_65ad_4384_8676_153772c7d8ea.slice/crio-5a9917ae8fe0fb5d2a427c70dad6fb087b7e7c4aaf6921b08e89f9604a069043 WatchSource:0}: Error finding container 5a9917ae8fe0fb5d2a427c70dad6fb087b7e7c4aaf6921b08e89f9604a069043: Status 404 returned error can't find the container with id 5a9917ae8fe0fb5d2a427c70dad6fb087b7e7c4aaf6921b08e89f9604a069043
Jul 28 11:11:17.330701 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:17.330602    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" event={"ID":"4311eeaa-65ad-4384-8676-153772c7d8ea","Type":"ContainerStarted","Data":"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"}
Jul 28 11:11:17.330701 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:17.330649    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" event={"ID":"4311eeaa-65ad-4384-8676-153772c7d8ea","Type":"ContainerStarted","Data":"5a9917ae8fe0fb5d2a427c70dad6fb087b7e7c4aaf6921b08e89f9604a069043"}
Jul 28 11:11:17.347804 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:17.347756    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" podStartSLOduration=1.347740269 podStartE2EDuration="1.347740269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:11:17.34580077 +0000 UTC m=+2902.623737945" watchObservedRunningTime="2026-07-28 11:11:17.347740269 +0000 UTC m=+2902.625677443"
Jul 28 11:11:20.040636 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:20.040592    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:11:26.975055 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:26.975010    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:26.975450 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:26.975068    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:26.987439 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:26.987417    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:27.400487 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:27.400459    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:11:46.145303 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:11:46.145266    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:12:07.243757 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:12:07.243721    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:12:18.848009 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:12:18.847934    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:12:41.844403 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:12:41.844367    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:12:55.426265 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:12:55.426231    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:12:55.433275 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:12:55.433255    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:13:00.149027 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:13:00.148990    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:13:27.841370 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:13:27.841333    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:13:44.248580 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:13:44.248529    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:14:04.550011 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:04.549921    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:14:20.741354 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:20.741312    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:14:41.638898 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:41.638863    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:14:55.534949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:55.534923    2544 scope.go:117] "RemoveContainer" containerID="5b01c58cb3a56691962473878888d54dc494b882e197362175863145163e2602"
Jul 28 11:14:55.543492 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:55.543476    2544 scope.go:117] "RemoveContainer" containerID="732e8e981d6929d5da33653b4761a0ff220696da7078540e6b2dae5c1f16fcd2"
Jul 28 11:14:59.248545 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:14:59.248510    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:15:00.134448 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.134415    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"]
Jul 28 11:15:00.139340 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.139319    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"
Jul 28 11:15:00.141778 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.141758    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:15:00.148583 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.148549    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"
Jul 28 11:15:00.153420 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.153397    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"]
Jul 28 11:15:00.287307 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.287282    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"]
Jul 28 11:15:00.289202 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:15:00.289179    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb589ee1e_c467_4f93_9441_acb9d991309b.slice/crio-ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92 WatchSource:0}: Error finding container ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92: Status 404 returned error can't find the container with id ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92
Jul 28 11:15:00.486739 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:00.486661    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c" event={"ID":"b589ee1e-c467-4f93-9441-acb9d991309b","Type":"ContainerStarted","Data":"ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92"}
Jul 28 11:15:01.492949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:01.492856    2544 generic.go:356] "Generic (PLEG): container finished" podID="b589ee1e-c467-4f93-9441-acb9d991309b" containerID="f038492c0933e4a20ce7f2a6835bb392ce1234f516ea2cade4f5773ff281a48b" exitCode=0
Jul 28 11:15:01.492949 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:01.492913    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c" event={"ID":"b589ee1e-c467-4f93-9441-acb9d991309b","Type":"ContainerDied","Data":"f038492c0933e4a20ce7f2a6835bb392ce1234f516ea2cade4f5773ff281a48b"}
Jul 28 11:15:02.635338 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:02.635314    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"
Jul 28 11:15:03.505392 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:03.505367    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"
Jul 28 11:15:03.505588 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:03.505401    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c" event={"ID":"b589ee1e-c467-4f93-9441-acb9d991309b","Type":"ContainerDied","Data":"ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92"}
Jul 28 11:15:03.505588 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:03.505458    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed47907efba9b54c2a312d5e38616569814c8b9147816e82110cc08ddea63b92"
Jul 28 11:15:22.040493 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:22.040414    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:15:28.228583 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:28.228522    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:15:37.344030 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:37.344000    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:15:43.252197 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.252143    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" podUID="4311eeaa-65ad-4384-8676-153772c7d8ea" containerName="main" containerID="cri-o://cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5" gracePeriod=45
Jul 28 11:15:43.516383 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.516329    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:15:43.603833 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.603806    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jfsx\" (UniqueName: \"kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.603999 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.603846    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.604079 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604032    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.604139 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604089    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.604198 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604148    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.604198 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604173    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache\") pod \"4311eeaa-65ad-4384-8676-153772c7d8ea\" (UID: \"4311eeaa-65ad-4384-8676-153772c7d8ea\") "
Jul 28 11:15:43.604324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604293    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home" (OuterVolumeSpecName: "home") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:15:43.604507 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604485    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-home\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.604507 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604476    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:15:43.604773 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.604673    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:15:43.606104 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.606074    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm" (OuterVolumeSpecName: "dshm") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:15:43.606198 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.606125    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:15:43.606310 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.606290    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx" (OuterVolumeSpecName: "kube-api-access-4jfsx") pod "4311eeaa-65ad-4384-8676-153772c7d8ea" (UID: "4311eeaa-65ad-4384-8676-153772c7d8ea"). InnerVolumeSpecName "kube-api-access-4jfsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:15:43.705954 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.705924    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jfsx\" (UniqueName: \"kubernetes.io/projected/4311eeaa-65ad-4384-8676-153772c7d8ea-kube-api-access-4jfsx\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.705954 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.705950    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-dshm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.705954 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.705961    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4311eeaa-65ad-4384-8676-153772c7d8ea-tls-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.706168 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.705969    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-tmp-dir\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.706168 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.705979    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4311eeaa-65ad-4384-8676-153772c7d8ea-model-cache\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:15:43.715602 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.715575    2544 generic.go:356] "Generic (PLEG): container finished" podID="4311eeaa-65ad-4384-8676-153772c7d8ea" containerID="cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5" exitCode=0
Jul 28 11:15:43.715742 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.715614    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" event={"ID":"4311eeaa-65ad-4384-8676-153772c7d8ea","Type":"ContainerDied","Data":"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"}
Jul 28 11:15:43.715742 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.715625    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"
Jul 28 11:15:43.715742 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.715648    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r" event={"ID":"4311eeaa-65ad-4384-8676-153772c7d8ea","Type":"ContainerDied","Data":"5a9917ae8fe0fb5d2a427c70dad6fb087b7e7c4aaf6921b08e89f9604a069043"}
Jul 28 11:15:43.715742 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.715669    2544 scope.go:117] "RemoveContainer" containerID="cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"
Jul 28 11:15:43.726078 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.726057    2544 scope.go:117] "RemoveContainer" containerID="cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"
Jul 28 11:15:43.726380 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:15:43.726359    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5\": container with ID starting with cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5 not found: ID does not exist" containerID="cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"
Jul 28 11:15:43.726465 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.726385    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5"} err="failed to get container status \"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5\": rpc error: code = NotFound desc = could not find container \"cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5\": container with ID starting with cfa6e755383748d8706f2e57c0d838416e75026371a56a59f3fcaf5bb880a9a5 not found: ID does not exist"
Jul 28 11:15:43.737834 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.737812    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:15:43.740763 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:43.740743    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/auth-test-model-683bdc-kserve-64d6c65f9f-j855r"]
Jul 28 11:15:45.270457 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:45.270418    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4311eeaa-65ad-4384-8676-153772c7d8ea" path="/var/lib/kubelet/pods/4311eeaa-65ad-4384-8676-153772c7d8ea/volumes"
Jul 28 11:15:50.145799 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:50.145754    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:15:55.569750 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:15:55.569723    2544 scope.go:117] "RemoveContainer" containerID="055eae97a257f311afcf28ca32b43c771dc8b4c056ea67d983a1cce6022dc428"
Jul 28 11:16:17.229379 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.229344    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:16:17.230023 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230003    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4311eeaa-65ad-4384-8676-153772c7d8ea" containerName="main"
Jul 28 11:16:17.230072 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230027    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="4311eeaa-65ad-4384-8676-153772c7d8ea" containerName="main"
Jul 28 11:16:17.230072 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230058    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b589ee1e-c467-4f93-9441-acb9d991309b" containerName="cleanup"
Jul 28 11:16:17.230072 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230068    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="b589ee1e-c467-4f93-9441-acb9d991309b" containerName="cleanup"
Jul 28 11:16:17.230189 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230165    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="b589ee1e-c467-4f93-9441-acb9d991309b" containerName="cleanup"
Jul 28 11:16:17.230189 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.230181    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="4311eeaa-65ad-4384-8676-153772c7d8ea" containerName="main"
Jul 28 11:16:17.233669 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.233649    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.237502 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.237482    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"openshift-service-ca.crt\""
Jul 28 11:16:17.237681 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.237493    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"default-dockercfg-k9r75\""
Jul 28 11:16:17.237773 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.237545    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"sub-test-model-683bdc-kserve-self-signed-certs\""
Jul 28 11:16:17.237840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.237671    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"kube-root-ca.crt\""
Jul 28 11:16:17.242900 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.242877    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:16:17.394054 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394022    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.394054 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394058    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.394294 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394075    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.394294 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394104    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx6cb\" (UniqueName: \"kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.394294 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394162    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.394396 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.394325    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495421 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495324    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx6cb\" (UniqueName: \"kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495421 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495375    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495467    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495525    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495573    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495601    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.495967 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.495938    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.496051 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.496024    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.496110 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.496054    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.498131 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.498108    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.498278 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.498180    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.503811 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.503791    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx6cb\" (UniqueName: \"kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb\") pod \"sub-test-model-683bdc-kserve-5844866bc5-knnbd\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") " pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.545545 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.545523    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:17.689265 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.688889    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:16:17.691861 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:16:17.691822    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2f40c38_fa1d_4bf9_ad82_6da1ceaaa2d4.slice/crio-8ef849014164a4e0deb0a4f2a1729c3cdf5dd035a1cf9915a8fbff4ba7969754 WatchSource:0}: Error finding container 8ef849014164a4e0deb0a4f2a1729c3cdf5dd035a1cf9915a8fbff4ba7969754: Status 404 returned error can't find the container with id 8ef849014164a4e0deb0a4f2a1729c3cdf5dd035a1cf9915a8fbff4ba7969754
Jul 28 11:16:17.910605 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.910549    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" event={"ID":"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4","Type":"ContainerStarted","Data":"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"}
Jul 28 11:16:17.910605 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.910610    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" event={"ID":"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4","Type":"ContainerStarted","Data":"8ef849014164a4e0deb0a4f2a1729c3cdf5dd035a1cf9915a8fbff4ba7969754"}
Jul 28 11:16:17.928497 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:17.928449    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" podStartSLOduration=0.928433461 podStartE2EDuration="928.433461ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:16:17.927165639 +0000 UTC m=+3203.205102807" watchObservedRunningTime="2026-07-28 11:16:17.928433461 +0000 UTC m=+3203.206370635"
Jul 28 11:16:20.143785 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:20.143742    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:16:27.545981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:27.545943    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:27.546510 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:27.546029    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:27.558464 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:27.558434    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:16:27.972350 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:16:27.972317    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:17:02.645750 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:02.645675    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:17:21.946099 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:21.946068    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:17:30.424867 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:30.424834    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:17:40.743018 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:40.742984    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:17:45.444171 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.444111    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" podUID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" containerName="main" containerID="cri-o://887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f" gracePeriod=45
Jul 28 11:17:45.695685 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.695626    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:17:45.780153 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780125    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780165    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780211    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780241    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx6cb\" (UniqueName: \"kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780288    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780533 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780341    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache\") pod \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\" (UID: \"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4\") "
Jul 28 11:17:45.780533 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780378    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home" (OuterVolumeSpecName: "home") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:17:45.780533 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780468    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:17:45.780696 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780614    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:17:45.780696 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780634    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-home\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:45.780696 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.780658    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tmp-dir\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:45.782431 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.782407    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm" (OuterVolumeSpecName: "dshm") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:17:45.782528 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.782472    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:17:45.782528 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.782480    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb" (OuterVolumeSpecName: "kube-api-access-wx6cb") pod "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" (UID: "f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4"). InnerVolumeSpecName "kube-api-access-wx6cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:17:45.881607 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.881554    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-model-cache\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:45.881607 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.881606    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-dshm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:45.881764 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.881619    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx6cb\" (UniqueName: \"kubernetes.io/projected/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-kube-api-access-wx6cb\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:45.881764 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:45.881635    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4-tls-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:17:46.358007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.357974    2544 generic.go:356] "Generic (PLEG): container finished" podID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" containerID="887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f" exitCode=0
Jul 28 11:17:46.358007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.358012    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"
Jul 28 11:17:46.358386 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.358062    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" event={"ID":"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4","Type":"ContainerDied","Data":"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"}
Jul 28 11:17:46.358386 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.358105    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd" event={"ID":"f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4","Type":"ContainerDied","Data":"8ef849014164a4e0deb0a4f2a1729c3cdf5dd035a1cf9915a8fbff4ba7969754"}
Jul 28 11:17:46.358386 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.358126    2544 scope.go:117] "RemoveContainer" containerID="887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"
Jul 28 11:17:46.367456 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.367441    2544 scope.go:117] "RemoveContainer" containerID="887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"
Jul 28 11:17:46.367770 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:17:46.367752    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f\": container with ID starting with 887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f not found: ID does not exist" containerID="887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"
Jul 28 11:17:46.367846 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.367775    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f"} err="failed to get container status \"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f\": rpc error: code = NotFound desc = could not find container \"887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f\": container with ID starting with 887ebf78936b9ddc02dcf3dfeb56ed1bc2c550c0311f3e710fcedb018999448f not found: ID does not exist"
Jul 28 11:17:46.378467 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.378445    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:17:46.383485 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:46.383464    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/sub-test-model-683bdc-kserve-5844866bc5-knnbd"]
Jul 28 11:17:47.270331 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:47.270297    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" path="/var/lib/kubelet/pods/f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4/volumes"
Jul 28 11:17:53.344380 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:53.344347    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:17:55.457138 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:55.457111    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:17:55.464319 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:17:55.464293    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:18:25.856016 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.855932    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:18:25.856471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.856376    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" containerName="main"
Jul 28 11:18:25.856471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.856391    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" containerName="main"
Jul 28 11:18:25.856471 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.856465    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f40c38-fa1d-4bf9-ad82-6da1ceaaa2d4" containerName="main"
Jul 28 11:18:25.859734 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.859718    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.862541 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.862516    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"openshift-service-ca.crt\""
Jul 28 11:18:25.862671 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.862572    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"default-dockercfg-k9r75\""
Jul 28 11:18:25.863684 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.863667    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"rate-test-model-683bdc-kserve-self-signed-certs\""
Jul 28 11:18:25.863753 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.863702    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-683bdc\"/\"kube-root-ca.crt\""
Jul 28 11:18:25.868749 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.868723    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:18:25.939894 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.939862    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.940070 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.939911    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.940070 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.939983    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.940070 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.940048    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.940206 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.940073    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgxsv\" (UniqueName: \"kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:25.940206 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:25.940095    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.041710 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041536    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.041710 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041601    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgxsv\" (UniqueName: \"kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.041710 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041639    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.041710 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041678    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.041710 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041719    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.042155 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.041787    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.042390 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.042364    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.042390 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.042383    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.042579 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.042484    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.045076 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.045052    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.045365 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.045343    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.050797 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.050772    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgxsv\" (UniqueName: \"kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv\") pod \"rate-test-model-683bdc-kserve-595f4f9947-6vwrb\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") " pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.169611 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.169515    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:26.309880 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.309746    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:18:26.312623 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:18:26.312596    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbce36297_56d0_4821_ac1a_c820c5304236.slice/crio-ee7eedf47f526fa00056130a8f0ad09f570708ed6b1df756088d3af490720cac WatchSource:0}: Error finding container ee7eedf47f526fa00056130a8f0ad09f570708ed6b1df756088d3af490720cac: Status 404 returned error can't find the container with id ee7eedf47f526fa00056130a8f0ad09f570708ed6b1df756088d3af490720cac
Jul 28 11:18:26.558901 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.558818    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" event={"ID":"bce36297-56d0-4821-ac1a-c820c5304236","Type":"ContainerStarted","Data":"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"}
Jul 28 11:18:26.558901 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.558859    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" event={"ID":"bce36297-56d0-4821-ac1a-c820c5304236","Type":"ContainerStarted","Data":"ee7eedf47f526fa00056130a8f0ad09f570708ed6b1df756088d3af490720cac"}
Jul 28 11:18:26.575778 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:26.575732    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" podStartSLOduration=1.575717405 podStartE2EDuration="1.575717405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:18:26.574388539 +0000 UTC m=+3331.852325715" watchObservedRunningTime="2026-07-28 11:18:26.575717405 +0000 UTC m=+3331.853654578"
Jul 28 11:18:29.355841 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:29.355796    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:18:36.170091 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:36.170064    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:36.170091 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:36.170104    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:36.182667 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:36.182647    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:18:36.624116 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:18:36.624089    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:19:02.238376 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:02.238342    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:19:24.547610 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:24.547575    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:19:32.080375 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:32.080342    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:19:40.246906 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:40.246871    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:19:43.447459 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.447426    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6975-140712451110928"]
Jul 28 11:19:43.450983 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.450967    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:43.453954 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.453929    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-l49gt\""
Jul 28 11:19:43.456999 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.456974    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6975-140712451110928"]
Jul 28 11:19:43.538923 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.538891    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2zgs\" (UniqueName: \"kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs\") pod \"test-curl-6975-140712451110928\" (UID: \"51610537-48ea-4a62-acb6-55cd395200d0\") " pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:43.640085 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.640056    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2zgs\" (UniqueName: \"kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs\") pod \"test-curl-6975-140712451110928\" (UID: \"51610537-48ea-4a62-acb6-55cd395200d0\") " pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:43.649519 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.649499    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2zgs\" (UniqueName: \"kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs\") pod \"test-curl-6975-140712451110928\" (UID: \"51610537-48ea-4a62-acb6-55cd395200d0\") " pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:43.761735 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.761671    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:43.896428 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.896397    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6975-140712451110928"]
Jul 28 11:19:43.897432 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:19:43.897407    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51610537_48ea_4a62_acb6_55cd395200d0.slice/crio-c869b18d63257bbb50b7ce29d585a98b566f93d5ffd0b711b64e5e0d216bac2b WatchSource:0}: Error finding container c869b18d63257bbb50b7ce29d585a98b566f93d5ffd0b711b64e5e0d216bac2b: Status 404 returned error can't find the container with id c869b18d63257bbb50b7ce29d585a98b566f93d5ffd0b711b64e5e0d216bac2b
Jul 28 11:19:43.898947 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.898926    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:19:43.958616 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:43.958586    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6975-140712451110928" event={"ID":"51610537-48ea-4a62-acb6-55cd395200d0","Type":"ContainerStarted","Data":"c869b18d63257bbb50b7ce29d585a98b566f93d5ffd0b711b64e5e0d216bac2b"}
Jul 28 11:19:44.965932 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:44.965896    2544 generic.go:356] "Generic (PLEG): container finished" podID="51610537-48ea-4a62-acb6-55cd395200d0" containerID="228a2fd59a9ab22405711ce53771ec5af25637feaff5a0fc225a0ed074fffc49" exitCode=0
Jul 28 11:19:44.966334 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:44.965993    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6975-140712451110928" event={"ID":"51610537-48ea-4a62-acb6-55cd395200d0","Type":"ContainerDied","Data":"228a2fd59a9ab22405711ce53771ec5af25637feaff5a0fc225a0ed074fffc49"}
Jul 28 11:19:46.113316 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.113294    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:46.147019 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.146996    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-6975-140712451110928_51610537-48ea-4a62-acb6-55cd395200d0/test-curl-6975-140712451110928/0.log"
Jul 28 11:19:46.157245 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.157222    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2zgs\" (UniqueName: \"kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs\") pod \"51610537-48ea-4a62-acb6-55cd395200d0\" (UID: \"51610537-48ea-4a62-acb6-55cd395200d0\") "
Jul 28 11:19:46.159420 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.159397    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs" (OuterVolumeSpecName: "kube-api-access-r2zgs") pod "51610537-48ea-4a62-acb6-55cd395200d0" (UID: "51610537-48ea-4a62-acb6-55cd395200d0"). InnerVolumeSpecName "kube-api-access-r2zgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:19:46.184354 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.184333    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6975-140712451110928"]
Jul 28 11:19:46.188040 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.188021    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6975-140712451110928"]
Jul 28 11:19:46.258012 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.257958    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2zgs\" (UniqueName: \"kubernetes.io/projected/51610537-48ea-4a62-acb6-55cd395200d0-kube-api-access-r2zgs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:46.978377 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.978304    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:46.978377 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:46.978346    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c869b18d63257bbb50b7ce29d585a98b566f93d5ffd0b711b64e5e0d216bac2b"
Jul 28 11:19:46.988302 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:19:46.988272    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-6975-140712451110928\" is forbidden: User \"system:node:ip-10-0-129-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-31.ec2.internal' and this object" podUID="51610537-48ea-4a62-acb6-55cd395200d0" pod="opendatahub/test-curl-6975-140712451110928"
Jul 28 11:19:47.100704 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.100660    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" podUID="bce36297-56d0-4821-ac1a-c820c5304236" containerName="main" containerID="cri-o://0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379" gracePeriod=45
Jul 28 11:19:47.271000 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.270920    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51610537-48ea-4a62-acb6-55cd395200d0" path="/var/lib/kubelet/pods/51610537-48ea-4a62-acb6-55cd395200d0/volumes"
Jul 28 11:19:47.354705 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.354685    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:19:47.366454 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366427    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366603 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366482    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366603 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366508    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366603 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366599    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgxsv\" (UniqueName: \"kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366652    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366696    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs\") pod \"bce36297-56d0-4821-ac1a-c820c5304236\" (UID: \"bce36297-56d0-4821-ac1a-c820c5304236\") "
Jul 28 11:19:47.366787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366712    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home" (OuterVolumeSpecName: "home") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:19:47.366924 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366782    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache" (OuterVolumeSpecName: "model-cache") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:19:47.366924 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.366835    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:19:47.367032 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.367017    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-home\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.367077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.367036    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-model-cache\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.367077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.367049    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-tmp-dir\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.368886 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.368851    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:19:47.368979 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.368931    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv" (OuterVolumeSpecName: "kube-api-access-dgxsv") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "kube-api-access-dgxsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:19:47.368979 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.368928    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm" (OuterVolumeSpecName: "dshm") pod "bce36297-56d0-4821-ac1a-c820c5304236" (UID: "bce36297-56d0-4821-ac1a-c820c5304236"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:19:47.468062 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.468021    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgxsv\" (UniqueName: \"kubernetes.io/projected/bce36297-56d0-4821-ac1a-c820c5304236-kube-api-access-dgxsv\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.468062 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.468058    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bce36297-56d0-4821-ac1a-c820c5304236-dshm\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.468062 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.468070    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bce36297-56d0-4821-ac1a-c820c5304236-tls-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:19:47.985807 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.985775    2544 generic.go:356] "Generic (PLEG): container finished" podID="bce36297-56d0-4821-ac1a-c820c5304236" containerID="0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379" exitCode=0
Jul 28 11:19:47.986003 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.985826    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"
Jul 28 11:19:47.986003 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.985845    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" event={"ID":"bce36297-56d0-4821-ac1a-c820c5304236","Type":"ContainerDied","Data":"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"}
Jul 28 11:19:47.986003 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.985954    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb" event={"ID":"bce36297-56d0-4821-ac1a-c820c5304236","Type":"ContainerDied","Data":"ee7eedf47f526fa00056130a8f0ad09f570708ed6b1df756088d3af490720cac"}
Jul 28 11:19:47.986003 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.985971    2544 scope.go:117] "RemoveContainer" containerID="0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"
Jul 28 11:19:47.999496 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.999470    2544 scope.go:117] "RemoveContainer" containerID="0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"
Jul 28 11:19:47.999880 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:19:47.999855    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379\": container with ID starting with 0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379 not found: ID does not exist" containerID="0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"
Jul 28 11:19:47.999945 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:47.999893    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379"} err="failed to get container status \"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379\": rpc error: code = NotFound desc = could not find container \"0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379\": container with ID starting with 0abae68a2db2e9dc546cfa7de8a0e746bfee8e0103018db514041c6e4d6dc379 not found: ID does not exist"
Jul 28 11:19:48.007420 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:48.007398    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:19:48.010922 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:48.010903    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-683bdc/rate-test-model-683bdc-kserve-595f4f9947-6vwrb"]
Jul 28 11:19:49.269480 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:19:49.269449    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bce36297-56d0-4821-ac1a-c820c5304236" path="/var/lib/kubelet/pods/bce36297-56d0-4821-ac1a-c820c5304236/volumes"
Jul 28 11:20:03.940029 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:03.939992    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:20:12.071391 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.071349    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:20:12.071982 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.071965    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bce36297-56d0-4821-ac1a-c820c5304236" containerName="main"
Jul 28 11:20:12.072034 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.071986    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="bce36297-56d0-4821-ac1a-c820c5304236" containerName="main"
Jul 28 11:20:12.072034 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.072001    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51610537-48ea-4a62-acb6-55cd395200d0" containerName="test-curl-6975-140712451110928"
Jul 28 11:20:12.072034 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.072010    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="51610537-48ea-4a62-acb6-55cd395200d0" containerName="test-curl-6975-140712451110928"
Jul 28 11:20:12.072139 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.072112    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="51610537-48ea-4a62-acb6-55cd395200d0" containerName="test-curl-6975-140712451110928"
Jul 28 11:20:12.072139 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.072127    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="bce36297-56d0-4821-ac1a-c820c5304236" containerName="main"
Jul 28 11:20:12.075382 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.075349    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.078030 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.078007    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-a93417-openshift-default-dockercfg-gmh97\""
Jul 28 11:20:12.087196 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.087170    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:20:12.176905 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.176874    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.176905 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.176907    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk4db\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177126 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.176929    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177126 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.176953    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177126 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.177011    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177126 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.177114    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177326 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.177159    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177326 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.177192    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.177326 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.177281    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278504 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278465    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278708 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278519    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278708 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278589    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278708 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278643    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278881 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278714    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278881 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278739    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4db\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278888    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.278981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278954    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279085 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278984    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279085 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.278999    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279085 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.279044    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279085 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.279070    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279279 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.279179    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.279416 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.279397    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.281223 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.281196    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.281481 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.281465    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.286353 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.286330    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.286883 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.286862    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk4db\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db\") pod \"e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") " pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.386836 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.386801    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:12.531700 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.531676    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:20:12.534433 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:20:12.534402    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba3abf8_49c0_451d_a96d_1796d84c3323.slice/crio-873b51723e537fdfc6e6f5eec493e6d49e3a9eaaffd443e051621110cb2df673 WatchSource:0}: Error finding container 873b51723e537fdfc6e6f5eec493e6d49e3a9eaaffd443e051621110cb2df673: Status 404 returned error can't find the container with id 873b51723e537fdfc6e6f5eec493e6d49e3a9eaaffd443e051621110cb2df673
Jul 28 11:20:12.536767 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.536727    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:12.536868 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.536819    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:12.536908 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:12.536867    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:13.117418 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:13.117384    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerStarted","Data":"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"}
Jul 28 11:20:13.117840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:13.117434    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerStarted","Data":"873b51723e537fdfc6e6f5eec493e6d49e3a9eaaffd443e051621110cb2df673"}
Jul 28 11:20:13.137896 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:13.137850    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podStartSLOduration=1.137834645 podStartE2EDuration="1.137834645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:20:13.136081655 +0000 UTC m=+3438.414018830" watchObservedRunningTime="2026-07-28 11:20:13.137834645 +0000 UTC m=+3438.415771819"
Jul 28 11:20:13.387199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:13.387124    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:13.393329 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:13.393301    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:14.122995 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:14.122970    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:14.123903 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:14.123886    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:17.848856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:17.848819    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:20:27.196235 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.196201    2544 generic.go:356] "Generic (PLEG): container finished" podID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerID="a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc" exitCode=0
Jul 28 11:20:27.196787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.196262    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerDied","Data":"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"}
Jul 28 11:20:27.196861 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.196809    2544 scope.go:117] "RemoveContainer" containerID="a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"
Jul 28 11:20:27.198042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.198015    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:27.198122 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.198081    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:27.198122 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:27.198113    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:20:28.207103 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:28.207054    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerStarted","Data":"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"}
Jul 28 11:20:28.387087 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:28.387058    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:28.393584 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:28.391676    2544 patch_prober.go:28] interesting pod/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 28 11:20:28.393584 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:28.391748    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 28 11:20:29.123250 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:29.123211    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:29.387178 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:29.387094    2544 patch_prober.go:28] interesting pod/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 28 11:20:29.387178 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:29.387155    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 28 11:20:30.387697 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:30.387665    2544 patch_prober.go:28] interesting pod/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 28 11:20:30.388237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:30.388206    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 28 11:20:31.387673 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:31.387637    2544 patch_prober.go:28] interesting pod/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 28 11:20:31.387950 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:31.387923    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 28 11:20:32.391341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:32.391313    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:32.392225 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:32.392206    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:20:52.937444 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:20:52.937407    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:21:23.452357 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:23.452268    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:21:27.427882 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.427851    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:27.431695 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.431675    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:27.439047 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.439020    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:27.455851 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.455828    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzbmt\" (UniqueName: \"kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt\") pod \"kuadrant-operator-catalog-sfm6x\" (UID: \"2723bafd-1f96-495a-968e-909b804d338f\") " pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:27.556875 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.556843    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzbmt\" (UniqueName: \"kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt\") pod \"kuadrant-operator-catalog-sfm6x\" (UID: \"2723bafd-1f96-495a-968e-909b804d338f\") " pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:27.566304 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.566277    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzbmt\" (UniqueName: \"kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt\") pod \"kuadrant-operator-catalog-sfm6x\" (UID: \"2723bafd-1f96-495a-968e-909b804d338f\") " pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:27.742979 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.742897    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:27.879396 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:27.879327    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:27.881748 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:21:27.881723    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2723bafd_1f96_495a_968e_909b804d338f.slice/crio-9ad4f8e5da9a328b2377f1e944efbf0d28855595db730b90b9bd77eca3e92ffc WatchSource:0}: Error finding container 9ad4f8e5da9a328b2377f1e944efbf0d28855595db730b90b9bd77eca3e92ffc: Status 404 returned error can't find the container with id 9ad4f8e5da9a328b2377f1e944efbf0d28855595db730b90b9bd77eca3e92ffc
Jul 28 11:21:28.513073 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:28.512978    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" event={"ID":"2723bafd-1f96-495a-968e-909b804d338f","Type":"ContainerStarted","Data":"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"}
Jul 28 11:21:28.513073 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:28.513030    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" event={"ID":"2723bafd-1f96-495a-968e-909b804d338f","Type":"ContainerStarted","Data":"9ad4f8e5da9a328b2377f1e944efbf0d28855595db730b90b9bd77eca3e92ffc"}
Jul 28 11:21:28.532207 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:28.532159    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" podStartSLOduration=1.211711647 podStartE2EDuration="1.532145165s" totalImagesPullingTime="320.433518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:21:28.529883297 +0000 UTC m=+3513.807820468" watchObservedRunningTime="2026-07-28 11:21:28.532145165 +0000 UTC m=+3513.810082338"
Jul 28 11:21:35.907360 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.907325    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:21:35.911074 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.911053    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.915113 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.915093    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-0a6eb5-openshift-default-dockercfg-lljnj\""
Jul 28 11:21:35.925958 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.925936    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:21:35.933711 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933679    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933797 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933732    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933841 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933793    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933841 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933826    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933906 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933860    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933906 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933887    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvxz\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.933980 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.933965    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.934043 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.934003    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:35.934080 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:35.934060    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035634 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035584    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035817 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035651    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035817 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035683    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035817 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035709    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvxz\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035817 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035764    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.035817 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035798    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035860    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035907    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.035940    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036077 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.036031    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036282 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.036201    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036282 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.036251    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036413 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.036393    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.036671 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.036652    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.038209 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.038188    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.038341 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.038323    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.051533 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.051510    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.057051 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.057023    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvxz\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz\") pod \"e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") " pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.222016 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.221944    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:36.377102 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.377070    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:21:36.378331 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:21:36.378306    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33f7d71_59da_49a3_959b_8120d6dccc1a.slice/crio-719324a50fd03543b9ef6791a4b46f1b73fa2a6dc370e1d026b0abc1bc01d0da WatchSource:0}: Error finding container 719324a50fd03543b9ef6791a4b46f1b73fa2a6dc370e1d026b0abc1bc01d0da: Status 404 returned error can't find the container with id 719324a50fd03543b9ef6791a4b46f1b73fa2a6dc370e1d026b0abc1bc01d0da
Jul 28 11:21:36.380577 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.380527    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:36.380677 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.380604    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:36.380677 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.380634    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:36.556129 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.556040    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerStarted","Data":"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"}
Jul 28 11:21:36.556129 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.556098    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerStarted","Data":"719324a50fd03543b9ef6791a4b46f1b73fa2a6dc370e1d026b0abc1bc01d0da"}
Jul 28 11:21:36.592836 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:36.592779    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podStartSLOduration=1.5927595380000001 podStartE2EDuration="1.592759538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:21:36.592376395 +0000 UTC m=+3521.870313569" watchObservedRunningTime="2026-07-28 11:21:36.592759538 +0000 UTC m=+3521.870696711"
Jul 28 11:21:37.222358 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.222331    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:37.227319 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.227298    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:37.561246 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.561175    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:37.561948 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.561931    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:37.743452 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.743423    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:37.743628 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.743512    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:37.765007 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:37.764986    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:38.586976 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:38.586951    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:38.669260 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:38.669229    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:40.576259 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:40.576210    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" podUID="2723bafd-1f96-495a-968e-909b804d338f" containerName="registry-server" containerID="cri-o://409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565" gracePeriod=2
Jul 28 11:21:40.828129 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:40.828076    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:40.879118 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:40.879089    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzbmt\" (UniqueName: \"kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt\") pod \"2723bafd-1f96-495a-968e-909b804d338f\" (UID: \"2723bafd-1f96-495a-968e-909b804d338f\") "
Jul 28 11:21:40.881281 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:40.881256    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt" (OuterVolumeSpecName: "kube-api-access-vzbmt") pod "2723bafd-1f96-495a-968e-909b804d338f" (UID: "2723bafd-1f96-495a-968e-909b804d338f"). InnerVolumeSpecName "kube-api-access-vzbmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:21:40.984586 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:40.981124    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzbmt\" (UniqueName: \"kubernetes.io/projected/2723bafd-1f96-495a-968e-909b804d338f-kube-api-access-vzbmt\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:21:41.581884 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.581798    2544 generic.go:356] "Generic (PLEG): container finished" podID="2723bafd-1f96-495a-968e-909b804d338f" containerID="409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565" exitCode=0
Jul 28 11:21:41.581884 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.581853    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x"
Jul 28 11:21:41.582336 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.581882    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" event={"ID":"2723bafd-1f96-495a-968e-909b804d338f","Type":"ContainerDied","Data":"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"}
Jul 28 11:21:41.582336 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.581935    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sfm6x" event={"ID":"2723bafd-1f96-495a-968e-909b804d338f","Type":"ContainerDied","Data":"9ad4f8e5da9a328b2377f1e944efbf0d28855595db730b90b9bd77eca3e92ffc"}
Jul 28 11:21:41.582336 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.581956    2544 scope.go:117] "RemoveContainer" containerID="409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"
Jul 28 11:21:41.594699 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.594675    2544 scope.go:117] "RemoveContainer" containerID="409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"
Jul 28 11:21:41.595122 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:21:41.595097    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565\": container with ID starting with 409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565 not found: ID does not exist" containerID="409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"
Jul 28 11:21:41.595241 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.595132    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565"} err="failed to get container status \"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565\": rpc error: code = NotFound desc = could not find container \"409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565\": container with ID starting with 409b8b6c073a89d68f8bb7d6c1b4d6cf4db15481e1fae0b23c2b9a79e3917565 not found: ID does not exist"
Jul 28 11:21:41.607722 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.607697    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:41.611447 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:41.611425    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sfm6x"]
Jul 28 11:21:43.270358 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:43.270328    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2723bafd-1f96-495a-968e-909b804d338f" path="/var/lib/kubelet/pods/2723bafd-1f96-495a-968e-909b804d338f/volumes"
Jul 28 11:21:48.961389 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:48.961358    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:21:54.231714 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.231635    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"]
Jul 28 11:21:54.232248 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.232228    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2723bafd-1f96-495a-968e-909b804d338f" containerName="registry-server"
Jul 28 11:21:54.232331 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.232250    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="2723bafd-1f96-495a-968e-909b804d338f" containerName="registry-server"
Jul 28 11:21:54.232382 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.232346    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="2723bafd-1f96-495a-968e-909b804d338f" containerName="registry-server"
Jul 28 11:21:54.236977 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.236955    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:54.239864 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.239843    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:21:54.239956 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.239941    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:21:54.246408 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.246381    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"]
Jul 28 11:21:54.316864 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.316832    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj\" (UID: \"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:54.417822 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.417790    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj\" (UID: \"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:54.418431 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.418414    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj\" (UID: \"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:54.549368 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.549290    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:54.707989 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:54.707966    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"]
Jul 28 11:21:54.710371 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:21:54.710340    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5ce47c_4e6c_4f6f_ba27_7b83c0769d77.slice/crio-f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122 WatchSource:0}: Error finding container f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122: Status 404 returned error can't find the container with id f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122
Jul 28 11:21:55.658636 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:55.658552    2544 generic.go:356] "Generic (PLEG): container finished" podID="1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" containerID="ff578c5d6c5dfd366c92a9004c06c9f030484d97c3598c30f78536de2841fbd0" exitCode=0
Jul 28 11:21:55.658907 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:55.658656    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj" event={"ID":"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77","Type":"ContainerDied","Data":"ff578c5d6c5dfd366c92a9004c06c9f030484d97c3598c30f78536de2841fbd0"}
Jul 28 11:21:55.658907 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:55.658708    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj" event={"ID":"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77","Type":"ContainerStarted","Data":"f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122"}
Jul 28 11:21:56.820130 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:56.820107    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:56.942153 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:56.942094    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca\") pod \"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77\" (UID: \"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77\") "
Jul 28 11:21:56.942408 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:56.942384    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" (UID: "1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:21:56.942494 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:56.942481    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:21:57.670403 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.670373    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"
Jul 28 11:21:57.670595 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.670396    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj" event={"ID":"1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77","Type":"ContainerDied","Data":"f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122"}
Jul 28 11:21:57.670595 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.670472    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f60070cfeb0afa7609df800869cfc532451526b07b6632dc2a7e59328aaf0122"
Jul 28 11:21:57.671981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.671957    2544 generic.go:356] "Generic (PLEG): container finished" podID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerID="ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb" exitCode=0
Jul 28 11:21:57.672105 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.672030    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerDied","Data":"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"}
Jul 28 11:21:57.672479 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.672459    2544 scope.go:117] "RemoveContainer" containerID="ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"
Jul 28 11:21:57.673702 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.673674    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:57.673792 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.673740    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:57.673792 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:57.673770    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 11:21:58.682883 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:58.682847    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerStarted","Data":"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"}
Jul 28 11:21:59.222611 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:59.222577    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:21:59.222790 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:59.222772    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 11:21:59.222845 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:21:59.222822    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 11:22:00.223089 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:00.223057    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 11:22:00.223475 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:00.223105    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 11:22:01.222505 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:01.222472    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 11:22:01.222693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:01.222520    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 11:22:02.222473 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:02.222431    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 11:22:02.222957 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:02.222492    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 11:22:03.226383 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:03.226354    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:22:03.226808 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:03.226613    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:22:03.227237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:03.227218    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:22:09.406973 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:09.406941    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"]
Jul 28 11:22:09.409775 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:09.409753    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-0a6eb5-657sj"]
Jul 28 11:22:11.269816 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:11.269783    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" path="/var/lib/kubelet/pods/1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77/volumes"
Jul 28 11:22:16.283239 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:16.283204    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:22:19.087599 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:19.087542    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:22:19.088061 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:19.087824    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy" containerID="cri-o://d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6" gracePeriod=30
Jul 28 11:22:22.860364 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.860332    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"]
Jul 28 11:22:22.860790 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.860775    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" containerName="revoke-keys"
Jul 28 11:22:22.860841 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.860791    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" containerName="revoke-keys"
Jul 28 11:22:22.860882 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.860873    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5ce47c-4e6c-4f6f-ba27-7b83c0769d77" containerName="revoke-keys"
Jul 28 11:22:22.865067 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.865052    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:22.871403 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.871377    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:22:22.872524 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.872505    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:22:22.874916 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.874874    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"]
Jul 28 11:22:22.976856 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:22.976815    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v\" (UID: \"17a15dc8-7beb-494e-9fe4-93d454ed560c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:23.078502 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.078460    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v\" (UID: \"17a15dc8-7beb-494e-9fe4-93d454ed560c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:23.079134 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.079110    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v\" (UID: \"17a15dc8-7beb-494e-9fe4-93d454ed560c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:23.176347 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.176262    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:23.325013 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.324990    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"]
Jul 28 11:22:23.327289 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:22:23.327266    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a15dc8_7beb_494e_9fe4_93d454ed560c.slice/crio-88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0 WatchSource:0}: Error finding container 88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0: Status 404 returned error can't find the container with id 88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0
Jul 28 11:22:23.820953 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.820865    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v" event={"ID":"17a15dc8-7beb-494e-9fe4-93d454ed560c","Type":"ContainerStarted","Data":"b1aa34c67bb0c9af701ef755529f8378af9ca325e680c68f75335535e670f8e1"}
Jul 28 11:22:23.820953 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.820922    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v" event={"ID":"17a15dc8-7beb-494e-9fe4-93d454ed560c","Type":"ContainerStarted","Data":"88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0"}
Jul 28 11:22:23.845407 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:23.845362    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v" podStartSLOduration=1.8453475080000001 podStartE2EDuration="1.845347508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:22:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:22:23.84021808 +0000 UTC m=+3569.118155255" watchObservedRunningTime="2026-07-28 11:22:23.845347508 +0000 UTC m=+3569.123284682"
Jul 28 11:22:24.351606 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.351526    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:22:24.392932 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.392903    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393107 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.392946    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393107 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.392973    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvvxz\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393107 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393046    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393107 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393079    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393327 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393108    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393327 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393160    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393327 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393201    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393327 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393229    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket\") pod \"a33f7d71-59da-49a3-959b-8120d6dccc1a\" (UID: \"a33f7d71-59da-49a3-959b-8120d6dccc1a\") "
Jul 28 11:22:24.393620 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393214    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:22:24.393704 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393608    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:22:24.393843 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393823    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:22:24.394013 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.393989    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:22:24.394165 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.394131    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:22:24.395916 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.395879    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz" (OuterVolumeSpecName: "kube-api-access-bvvxz") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "kube-api-access-bvvxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:22:24.396144 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.396123    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:22:24.396295 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.396277    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:22:24.396462 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.396446    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a33f7d71-59da-49a3-959b-8120d6dccc1a" (UID: "a33f7d71-59da-49a3-959b-8120d6dccc1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:22:24.495082 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495047    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495082 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495076    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495082 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495085    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495093    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495102    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495109    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvvxz\" (UniqueName: \"kubernetes.io/projected/a33f7d71-59da-49a3-959b-8120d6dccc1a-kube-api-access-bvvxz\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495118    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a33f7d71-59da-49a3-959b-8120d6dccc1a-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495125    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.495324 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.495132    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a33f7d71-59da-49a3-959b-8120d6dccc1a-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:24.829125 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.829035    2544 generic.go:356] "Generic (PLEG): container finished" podID="17a15dc8-7beb-494e-9fe4-93d454ed560c" containerID="b1aa34c67bb0c9af701ef755529f8378af9ca325e680c68f75335535e670f8e1" exitCode=0
Jul 28 11:22:24.829275 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.829117    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v" event={"ID":"17a15dc8-7beb-494e-9fe4-93d454ed560c","Type":"ContainerDied","Data":"b1aa34c67bb0c9af701ef755529f8378af9ca325e680c68f75335535e670f8e1"}
Jul 28 11:22:24.830812 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.830784    2544 generic.go:356] "Generic (PLEG): container finished" podID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerID="d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6" exitCode=0
Jul 28 11:22:24.830936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.830821    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerDied","Data":"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"}
Jul 28 11:22:24.830936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.830833    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"
Jul 28 11:22:24.830936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.830863    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2" event={"ID":"a33f7d71-59da-49a3-959b-8120d6dccc1a","Type":"ContainerDied","Data":"719324a50fd03543b9ef6791a4b46f1b73fa2a6dc370e1d026b0abc1bc01d0da"}
Jul 28 11:22:24.830936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.830885    2544 scope.go:117] "RemoveContainer" containerID="d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"
Jul 28 11:22:24.841192 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.841169    2544 scope.go:117] "RemoveContainer" containerID="ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"
Jul 28 11:22:24.850463 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.850443    2544 scope.go:117] "RemoveContainer" containerID="d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"
Jul 28 11:22:24.850786 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:22:24.850767    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6\": container with ID starting with d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6 not found: ID does not exist" containerID="d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"
Jul 28 11:22:24.850857 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.850796    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6"} err="failed to get container status \"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6\": rpc error: code = NotFound desc = could not find container \"d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6\": container with ID starting with d996f45614ead98bba0d8b8e0a30deaf429a97026bfdbe47e27691a2a46d0de6 not found: ID does not exist"
Jul 28 11:22:24.850857 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.850812    2544 scope.go:117] "RemoveContainer" containerID="ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"
Jul 28 11:22:24.851086 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:22:24.851067    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb\": container with ID starting with ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb not found: ID does not exist" containerID="ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"
Jul 28 11:22:24.851137 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.851092    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb"} err="failed to get container status \"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb\": rpc error: code = NotFound desc = could not find container \"ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb\": container with ID starting with ccd4866c844fbbd5b5d14b42fdab33642329183bea23787d727abe7e2891c7eb not found: ID does not exist"
Jul 28 11:22:24.859931 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.859912    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:22:24.871934 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:24.871913    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0a6eb5-openshift-default-867c5db9fd-2ldk2"]
Jul 28 11:22:25.270953 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:25.270919    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" path="/var/lib/kubelet/pods/a33f7d71-59da-49a3-959b-8120d6dccc1a/volumes"
Jul 28 11:22:25.998122 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:25.998098    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:26.111003 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.110977    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca\") pod \"17a15dc8-7beb-494e-9fe4-93d454ed560c\" (UID: \"17a15dc8-7beb-494e-9fe4-93d454ed560c\") "
Jul 28 11:22:26.111317 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.111295    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17a15dc8-7beb-494e-9fe4-93d454ed560c" (UID: "17a15dc8-7beb-494e-9fe4-93d454ed560c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:22:26.211818 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.211783    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a15dc8-7beb-494e-9fe4-93d454ed560c-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:22:26.843091 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.843063    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"
Jul 28 11:22:26.843091 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.843071    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v" event={"ID":"17a15dc8-7beb-494e-9fe4-93d454ed560c","Type":"ContainerDied","Data":"88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0"}
Jul 28 11:22:26.843305 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:26.843116    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88585896e14039f5d9c399e97fc197e91df5453e8eb5db141650c77a9aa797d0"
Jul 28 11:22:39.851749 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:39.851714    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:22:43.156519 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:43.156483    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"]
Jul 28 11:22:43.168288 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:43.168261    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1c9694-2vm6v"]
Jul 28 11:22:43.269644 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:43.269605    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17a15dc8-7beb-494e-9fe4-93d454ed560c" path="/var/lib/kubelet/pods/17a15dc8-7beb-494e-9fe4-93d454ed560c/volumes"
Jul 28 11:22:54.652243 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:54.652199    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:22:55.489424 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:55.489395    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:22:55.498936 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:22:55.498913    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:23:33.582754 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.582722    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"]
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583115    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a15dc8-7beb-494e-9fe4-93d454ed560c" containerName="revoke-keys"
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583128    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a15dc8-7beb-494e-9fe4-93d454ed560c" containerName="revoke-keys"
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583147    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583152    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583159    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.583199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583164    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.583425 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583224    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.583425 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583234    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="17a15dc8-7beb-494e-9fe4-93d454ed560c" containerName="revoke-keys"
Jul 28 11:23:33.583425 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.583240    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33f7d71-59da-49a3-959b-8120d6dccc1a" containerName="istio-proxy"
Jul 28 11:23:33.586516 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.586500    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:33.589192 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.589167    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gvrfc\""
Jul 28 11:23:33.589325 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.589251    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 11:23:33.597160 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.597132    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"]
Jul 28 11:23:33.727782 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.727743    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t\" (UID: \"881748c9-6ceb-40ed-a379-4bf72e356ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:33.828199 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.828166    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t\" (UID: \"881748c9-6ceb-40ed-a379-4bf72e356ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:33.828815 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.828795    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t\" (UID: \"881748c9-6ceb-40ed-a379-4bf72e356ac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:33.899874 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:33.899848    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:34.037530 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:34.037503    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"]
Jul 28 11:23:34.039639 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:23:34.039614    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881748c9_6ceb_40ed_a379_4bf72e356ac9.slice/crio-0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae WatchSource:0}: Error finding container 0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae: Status 404 returned error can't find the container with id 0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae
Jul 28 11:23:34.206948 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:34.206885    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t" event={"ID":"881748c9-6ceb-40ed-a379-4bf72e356ac9","Type":"ContainerStarted","Data":"0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae"}
Jul 28 11:23:35.213803 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:35.213764    2544 generic.go:356] "Generic (PLEG): container finished" podID="881748c9-6ceb-40ed-a379-4bf72e356ac9" containerID="a83c7ed470ef619fce11aa5c717da6f6eec286a76747e1c5bf8415bb58c42f44" exitCode=0
Jul 28 11:23:35.214201 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:35.213851    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t" event={"ID":"881748c9-6ceb-40ed-a379-4bf72e356ac9","Type":"ContainerDied","Data":"a83c7ed470ef619fce11aa5c717da6f6eec286a76747e1c5bf8415bb58c42f44"}
Jul 28 11:23:36.360234 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:36.360214    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:36.450840 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:36.450811    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca\") pod \"881748c9-6ceb-40ed-a379-4bf72e356ac9\" (UID: \"881748c9-6ceb-40ed-a379-4bf72e356ac9\") "
Jul 28 11:23:36.451157 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:36.451137    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "881748c9-6ceb-40ed-a379-4bf72e356ac9" (UID: "881748c9-6ceb-40ed-a379-4bf72e356ac9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:23:36.552198 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:36.552136    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881748c9-6ceb-40ed-a379-4bf72e356ac9-maas-api-service-ca\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:23:37.225480 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:37.225451    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"
Jul 28 11:23:37.225651 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:37.225450    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t" event={"ID":"881748c9-6ceb-40ed-a379-4bf72e356ac9","Type":"ContainerDied","Data":"0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae"}
Jul 28 11:23:37.225651 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:37.225589    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a7ee7f6719bc0826c95547d1cf1d8d8218bd8a346c9066827d1e8b3ffc778ae"
Jul 28 11:23:48.079521 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:48.079485    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:23:48.669712 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:48.669680    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"]
Jul 28 11:23:48.675733 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:48.675710    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a93417-5lx2t"]
Jul 28 11:23:49.270271 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:49.270229    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881748c9-6ceb-40ed-a379-4bf72e356ac9" path="/var/lib/kubelet/pods/881748c9-6ceb-40ed-a379-4bf72e356ac9/volumes"
Jul 28 11:23:56.054552 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:56.054517    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:23:56.055027 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:23:56.054751    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy" containerID="cri-o://70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c" gracePeriod=30
Jul 28 11:24:01.308410 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.308385    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:24:01.351254 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.351222    2544 generic.go:356] "Generic (PLEG): container finished" podID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerID="70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c" exitCode=0
Jul 28 11:24:01.351381 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.351261    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"
Jul 28 11:24:01.351381 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.351300    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerDied","Data":"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"}
Jul 28 11:24:01.351381 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.351348    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv" event={"ID":"bba3abf8-49c0-451d-a96d-1796d84c3323","Type":"ContainerDied","Data":"873b51723e537fdfc6e6f5eec493e6d49e3a9eaaffd443e051621110cb2df673"}
Jul 28 11:24:01.351381 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.351377    2544 scope.go:117] "RemoveContainer" containerID="70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"
Jul 28 11:24:01.360641 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.360624    2544 scope.go:117] "RemoveContainer" containerID="a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"
Jul 28 11:24:01.368782 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.368766    2544 scope.go:117] "RemoveContainer" containerID="70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"
Jul 28 11:24:01.369059 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:24:01.369039    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c\": container with ID starting with 70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c not found: ID does not exist" containerID="70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"
Jul 28 11:24:01.369115 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.369075    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c"} err="failed to get container status \"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c\": rpc error: code = NotFound desc = could not find container \"70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c\": container with ID starting with 70c17b288941da87cb6d6203c2e52e53dfdfb53d5a74a499a753141e4a62932c not found: ID does not exist"
Jul 28 11:24:01.369115 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.369091    2544 scope.go:117] "RemoveContainer" containerID="a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"
Jul 28 11:24:01.369313 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:24:01.369294    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc\": container with ID starting with a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc not found: ID does not exist" containerID="a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"
Jul 28 11:24:01.369360 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.369318    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc"} err="failed to get container status \"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc\": rpc error: code = NotFound desc = could not find container \"a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc\": container with ID starting with a4489c832aae8609d0aef2f92ca52c2e7d06dab9793382e550bdf197095d0bbc not found: ID does not exist"
Jul 28 11:24:01.378067 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378044    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378181 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378077    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378181 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378120    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378181 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378142    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378184    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378213    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378239    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378283    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378307    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk4db\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db\") pod \"bba3abf8-49c0-451d-a96d-1796d84c3323\" (UID: \"bba3abf8-49c0-451d-a96d-1796d84c3323\") "
Jul 28 11:24:01.378351 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378322    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:01.378683 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378440    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:01.378683 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378614    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:01.378801 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378774    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.378864 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378799    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.378864 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.378819    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.379314 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.379287    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data" (OuterVolumeSpecName: "istio-data") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:01.379405 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.379338    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:24:01.380517 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.380496    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:01.380684 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.380653    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db" (OuterVolumeSpecName: "kube-api-access-dk4db") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "kube-api-access-dk4db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:24:01.380769 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.380730    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token" (OuterVolumeSpecName: "istio-token") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:24:01.380769 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.380741    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bba3abf8-49c0-451d-a96d-1796d84c3323" (UID: "bba3abf8-49c0-451d-a96d-1796d84c3323"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:24:01.480273 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480238    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bba3abf8-49c0-451d-a96d-1796d84c3323-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.480273 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480265    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.480273 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480277    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.480483 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480288    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk4db\" (UniqueName: \"kubernetes.io/projected/bba3abf8-49c0-451d-a96d-1796d84c3323-kube-api-access-dk4db\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.480483 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480301    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.480483 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.480312    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bba3abf8-49c0-451d-a96d-1796d84c3323-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:01.674429 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.674401    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:24:01.678660 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:01.678637    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-a93417-openshift-default-b56d9c654-xxgxv"]
Jul 28 11:24:03.270165 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:03.270133    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" path="/var/lib/kubelet/pods/bba3abf8-49c0-451d-a96d-1796d84c3323/volumes"
Jul 28 11:24:10.753115 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:10.753077    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:24:30.938527 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:30.938429    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"]
Jul 28 11:24:30.946693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:30.946661    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-683bdc-29753955-2xm4c"]
Jul 28 11:24:31.269745 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:31.269667    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b589ee1e-c467-4f93-9441-acb9d991309b" path="/var/lib/kubelet/pods/b589ee1e-c467-4f93-9441-acb9d991309b/volumes"
Jul 28 11:24:33.671990 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:33.671958    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jmkpl"]
Jul 28 11:24:42.018016 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:42.017976    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:24:42.018641 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:42.018245    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy" containerID="cri-o://d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea" gracePeriod=30
Jul 28 11:24:47.276122 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.276100    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:24:47.374846 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374822    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.374981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374871    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.374981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374895    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.374981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374917    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.374981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374952    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.374981 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.374981    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqb4p\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.375180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375003    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.375180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375038    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.375180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375129    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:47.375180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375154    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:47.375180 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375171    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs\") pod \"294fbe09-9481-4814-a879-ca4f48058f23\" (UID: \"294fbe09-9481-4814-a879-ca4f48058f23\") "
Jul 28 11:24:47.375489 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375468    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-credential-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.375705 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375689    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-socket\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.375944 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375583    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:24:47.376120 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.375710    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:47.376120 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.376030    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data" (OuterVolumeSpecName: "istio-data") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:47.377382 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.377354    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p" (OuterVolumeSpecName: "kube-api-access-tqb4p") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "kube-api-access-tqb4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:24:47.377469 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.377404    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token" (OuterVolumeSpecName: "istio-token") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:24:47.377999 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.377962    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 11:24:47.378093 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.378009    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "294fbe09-9481-4814-a879-ca4f48058f23" (UID: "294fbe09-9481-4814-a879-ca4f48058f23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:24:47.476911 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476886    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-istio-token\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.476911 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476911    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqb4p\" (UniqueName: \"kubernetes.io/projected/294fbe09-9481-4814-a879-ca4f48058f23-kube-api-access-tqb4p\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.477060 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476921    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294fbe09-9481-4814-a879-ca4f48058f23-istio-podinfo\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.477060 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476931    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-data\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.477060 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476938    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-workload-certs\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.477060 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476945    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294fbe09-9481-4814-a879-ca4f48058f23-istio-envoy\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.477060 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.476952    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294fbe09-9481-4814-a879-ca4f48058f23-istiod-ca-cert\") on node \"ip-10-0-129-31.ec2.internal\" DevicePath \"\""
Jul 28 11:24:47.593116 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.593084    2544 generic.go:356] "Generic (PLEG): container finished" podID="294fbe09-9481-4814-a879-ca4f48058f23" containerID="d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea" exitCode=0
Jul 28 11:24:47.593302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.593153    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerDied","Data":"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"}
Jul 28 11:24:47.593302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.593167    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"
Jul 28 11:24:47.593302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.593222    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd" event={"ID":"294fbe09-9481-4814-a879-ca4f48058f23","Type":"ContainerDied","Data":"8274e5d3f1e05dd62c9a8bee6792ccb1ae1877dafef6958f1ba90c88f7f6f805"}
Jul 28 11:24:47.593302 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.593244    2544 scope.go:117] "RemoveContainer" containerID="d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"
Jul 28 11:24:47.603972 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.603949    2544 scope.go:117] "RemoveContainer" containerID="9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"
Jul 28 11:24:47.612223 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.612208    2544 scope.go:117] "RemoveContainer" containerID="d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"
Jul 28 11:24:47.612478 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:24:47.612459    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea\": container with ID starting with d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea not found: ID does not exist" containerID="d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"
Jul 28 11:24:47.612549 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.612489    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea"} err="failed to get container status \"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea\": rpc error: code = NotFound desc = could not find container \"d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea\": container with ID starting with d9cfbdacfa507ac1ef57af15d4888a180aea590d9725d5aa2d796aa57bae72ea not found: ID does not exist"
Jul 28 11:24:47.612549 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.612512    2544 scope.go:117] "RemoveContainer" containerID="9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"
Jul 28 11:24:47.612788 ip-10-0-129-31 kubenswrapper[2544]: E0728 11:24:47.612771    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5\": container with ID starting with 9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5 not found: ID does not exist" containerID="9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"
Jul 28 11:24:47.612844 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.612796    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5"} err="failed to get container status \"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5\": rpc error: code = NotFound desc = could not find container \"9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5\": container with ID starting with 9ac38dc611722da66368dd9f21056dabd8b971a0b99815786b9d914e85172aa5 not found: ID does not exist"
Jul 28 11:24:47.622953 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.622935    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:24:47.627345 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:47.627300    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-683bdc-openshift-default-546d9c9548-f4jjd"]
Jul 28 11:24:48.994576 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:48.994524    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9whhf_f0f135df-5161-4f47-b431-8dca8b419610/manager/0.log"
Jul 28 11:24:49.123720 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.123693    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5k7t4_f8cda8cc-33ad-41e4-bd33-3a8f66243322/manager/0.log"
Jul 28 11:24:49.269805 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.269734    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="294fbe09-9481-4814-a879-ca4f48058f23" path="/var/lib/kubelet/pods/294fbe09-9481-4814-a879-ca4f48058f23/volumes"
Jul 28 11:24:49.396951 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.396921    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5mh2r_a8c05a0a-bbec-46f7-bb87-aaa8685a5b77/server/0.log"
Jul 28 11:24:49.537961 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.537874    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zfj2q_9d0cc820-c243-41de-8bf8-d8b17b3c36f8/manager/0.log"
Jul 28 11:24:49.681783 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.681754    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f66f7dc-hb2m2_953e374c-964b-40a6-8ec5-3dd076748f39/copy-manifests/0.log"
Jul 28 11:24:49.691928 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:49.691905    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f66f7dc-hb2m2_953e374c-964b-40a6-8ec5-3dd076748f39/manager/0.log"
Jul 28 11:24:52.759703 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:52.759669    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2jd9_66c8f60a-adc7-4198-bfa4-9f51eac3bf19/registry-server/0.log"
Jul 28 11:24:52.888809 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:52.888779    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c8lc9_5a98b69b-fb68-4085-8857-c1b6bb665ea5/manager/0.log"
Jul 28 11:24:53.017549 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:53.017466    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jmkpl_d9fa5e54-77d8-4fe8-903a-82ccf50e6a11/limitador/0.log"
Jul 28 11:24:53.150281 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:53.150250    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mh7qv_b5078561-31b2-4c06-96af-d1142fc8f1d2/manager/0.log"
Jul 28 11:24:53.520732 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:53.520702    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-77785_035a1d4f-39a7-4f52-94c3-2ac37fdaed68/istio-proxy/0.log"
Jul 28 11:24:53.899529 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:53.899502    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d6b86846-rkl5w_ca029121-a68d-4531-9152-23bf1a84c263/kube-auth-proxy/0.log"
Jul 28 11:24:54.050628 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:54.050598    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/1.log"
Jul 28 11:24:54.431794 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:54.431758    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d49bfb5-n7bxk_435525f2-d692-4697-b299-0a6484bbd281/router/0.log"
Jul 28 11:24:55.170794 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:55.170761    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2lv9m_1a2bb213-c041-45ae-8311-46380a9cd3a9/main/0.log"
Jul 28 11:24:55.421929 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:55.421853    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9kvc7_c99e0cf6-ce79-4e50-a8d8-afae068c7114/main/0.log"
Jul 28 11:24:55.673695 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:24:55.673611    2544 scope.go:117] "RemoveContainer" containerID="f038492c0933e4a20ce7f2a6835bb392ce1234f516ea2cade4f5773ff281a48b"
Jul 28 11:25:04.312578 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:04.312516    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-67ksr_0cefb3d3-e521-4c48-a063-77eb91f8ef91/global-pull-secret-syncer/0.log"
Jul 28 11:25:04.544368 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:04.544340    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gh8sh_603776a9-6511-4a16-9f19-0b33fa8c6772/konnectivity-agent/0.log"
Jul 28 11:25:04.618270 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:04.618214    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-31.ec2.internal_85547fd26d62ca170d8549b389f25e88/haproxy/0.log"
Jul 28 11:25:10.099773 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:10.099733    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2jd9_66c8f60a-adc7-4198-bfa4-9f51eac3bf19/registry-server/0.log"
Jul 28 11:25:10.269502 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:10.269467    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c8lc9_5a98b69b-fb68-4085-8857-c1b6bb665ea5/manager/0.log"
Jul 28 11:25:10.318858 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:10.318828    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jmkpl_d9fa5e54-77d8-4fe8-903a-82ccf50e6a11/limitador/0.log"
Jul 28 11:25:10.433346 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:10.433319    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mh7qv_b5078561-31b2-4c06-96af-d1142fc8f1d2/manager/0.log"
Jul 28 11:25:12.384787 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:12.384756    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-dd6jz_daebe161-b184-41a5-9bb6-e41424e14559/cluster-monitoring-operator/0.log"
Jul 28 11:25:12.559948 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:12.559916    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25pz_c77abb78-4a3c-4dd4-afbc-fc34770abee5/node-exporter/0.log"
Jul 28 11:25:12.579780 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:12.579756    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25pz_c77abb78-4a3c-4dd4-afbc-fc34770abee5/kube-rbac-proxy/0.log"
Jul 28 11:25:12.605903 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:12.605877    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25pz_c77abb78-4a3c-4dd4-afbc-fc34770abee5/init-textfile/0.log"
Jul 28 11:25:13.174011 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:13.173974    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-9ws4c_09292749-a301-4d15-ab37-48683a5f1949/prometheus-operator-admission-webhook/0.log"
Jul 28 11:25:14.395530 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395493    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"]
Jul 28 11:25:14.395918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395897    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881748c9-6ceb-40ed-a379-4bf72e356ac9" containerName="revoke-keys"
Jul 28 11:25:14.395918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395910    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="881748c9-6ceb-40ed-a379-4bf72e356ac9" containerName="revoke-keys"
Jul 28 11:25:14.395918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395919    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395925    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395941    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395946    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395952    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395957    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395968    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.395974    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.396034    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396042 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.396041    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="bba3abf8-49c0-451d-a96d-1796d84c3323" containerName="istio-proxy"
Jul 28 11:25:14.396371 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.396053    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="294fbe09-9481-4814-a879-ca4f48058f23" containerName="istio-proxy"
Jul 28 11:25:14.396371 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.396062    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="881748c9-6ceb-40ed-a379-4bf72e356ac9" containerName="revoke-keys"
Jul 28 11:25:14.399059 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.399044    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.401401 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.401381    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zgn8k\"/\"default-dockercfg-j5kdx\""
Jul 28 11:25:14.401532 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.401434    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgn8k\"/\"openshift-service-ca.crt\""
Jul 28 11:25:14.402371 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.402352    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgn8k\"/\"kube-root-ca.crt\""
Jul 28 11:25:14.409665 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.409643    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"]
Jul 28 11:25:14.500464 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.500430    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-lib-modules\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.500665 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.500494    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-proc\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.500665 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.500598    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mlz\" (UniqueName: \"kubernetes.io/projected/a91ff121-ba0f-4886-ab81-f20ffadece46-kube-api-access-q6mlz\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.500665 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.500659    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-podres\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.500781 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.500697    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-sys\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.593915 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.593883    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-pnm92_bb033949-9ce9-4b56-a7e9-7f43383d5af3/networking-console-plugin/0.log"
Jul 28 11:25:14.601362 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601335    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-lib-modules\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601494 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601389    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-proc\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601494 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601424    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mlz\" (UniqueName: \"kubernetes.io/projected/a91ff121-ba0f-4886-ab81-f20ffadece46-kube-api-access-q6mlz\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601494 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601468    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-podres\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601664 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601499    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-sys\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601664 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601531    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-lib-modules\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601664 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601537    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-proc\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601664 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601597    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-podres\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.601664 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.601605    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a91ff121-ba0f-4886-ab81-f20ffadece46-sys\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.609960 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.609939    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mlz\" (UniqueName: \"kubernetes.io/projected/a91ff121-ba0f-4886-ab81-f20ffadece46-kube-api-access-q6mlz\") pod \"perf-node-gather-daemonset-549n2\" (UID: \"a91ff121-ba0f-4886-ab81-f20ffadece46\") " pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.709700 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.709629    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:14.848934 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:14.848911    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"]
Jul 28 11:25:14.851148 ip-10-0-129-31 kubenswrapper[2544]: W0728 11:25:14.851120    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda91ff121_ba0f_4886_ab81_f20ffadece46.slice/crio-ebf059ae9c563480cc9909d430908d4c369e1a326152f6dc755062c4d6347889 WatchSource:0}: Error finding container ebf059ae9c563480cc9909d430908d4c369e1a326152f6dc755062c4d6347889: Status 404 returned error can't find the container with id ebf059ae9c563480cc9909d430908d4c369e1a326152f6dc755062c4d6347889
Jul 28 11:25:15.242978 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:15.242946    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/2.log"
Jul 28 11:25:15.250150 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:15.250122    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bmpbj_b5ddfb23-d3c5-412f-a084-79ece8f6ac37/console-operator/3.log"
Jul 28 11:25:15.738693 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:15.738662    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2" event={"ID":"a91ff121-ba0f-4886-ab81-f20ffadece46","Type":"ContainerStarted","Data":"d7fc9dbaddc9901176874f2b78e00f14064f9c72bf175fce138187b620bd72ab"}
Jul 28 11:25:15.739076 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:15.738827    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:15.739076 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:15.738870    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2" event={"ID":"a91ff121-ba0f-4886-ab81-f20ffadece46","Type":"ContainerStarted","Data":"ebf059ae9c563480cc9909d430908d4c369e1a326152f6dc755062c4d6347889"}
Jul 28 11:25:16.482020 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:16.481990    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-fd7cx_eae859ae-347d-4ba4-89c0-19470bfa944b/volume-data-source-validator/0.log"
Jul 28 11:25:17.652911 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:17.652880    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xkrf6_95553483-b6f2-47e4-a9dd-02266be7a816/dns/0.log"
Jul 28 11:25:17.672478 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:17.672451    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xkrf6_95553483-b6f2-47e4-a9dd-02266be7a816/kube-rbac-proxy/0.log"
Jul 28 11:25:17.695323 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:17.695298    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwn8m_ac56291e-dfd0-4c80-b2d8-f246abe4fa4b/dns-node-resolver/0.log"
Jul 28 11:25:18.427993 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:18.427958    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p4g9v_5e953407-ddc7-4913-950e-9e4048297038/node-ca/0.log"
Jul 28 11:25:19.369918 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:19.369887    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-77785_035a1d4f-39a7-4f52-94c3-2ac37fdaed68/istio-proxy/0.log"
Jul 28 11:25:19.561048 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:19.561021    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d6b86846-rkl5w_ca029121-a68d-4531-9152-23bf1a84c263/kube-auth-proxy/0.log"
Jul 28 11:25:19.592821 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:19.592789    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/0.log"
Jul 28 11:25:19.646690 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:19.646607    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/1.log"
Jul 28 11:25:19.728509 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:19.728478    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d49bfb5-n7bxk_435525f2-d692-4697-b299-0a6484bbd281/router/0.log"
Jul 28 11:25:20.592705 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:20.592669    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tgdp6_c2378764-8651-4edf-b937-f2738f9f2236/serve-healthcheck-canary/0.log"
Jul 28 11:25:21.100577 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.100540    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-gg27m_3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122/insights-operator/0.log"
Jul 28 11:25:21.102721 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.102700    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-gg27m_3b788e7c-ecd7-4aa5-bfcb-0a1ad1d97122/insights-operator/1.log"
Jul 28 11:25:21.192031 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.192003    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nsnq9_321c2faa-5209-4b2f-ae32-4e3bb51f1b38/kube-rbac-proxy/0.log"
Jul 28 11:25:21.210832 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.210809    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nsnq9_321c2faa-5209-4b2f-ae32-4e3bb51f1b38/exporter/0.log"
Jul 28 11:25:21.233227 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.233206    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nsnq9_321c2faa-5209-4b2f-ae32-4e3bb51f1b38/extractor/0.log"
Jul 28 11:25:21.754167 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.754140    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2"
Jul 28 11:25:21.770751 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:21.770706    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zgn8k/perf-node-gather-daemonset-549n2" podStartSLOduration=7.770692455 podStartE2EDuration="7.770692455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:25:15.765628961 +0000 UTC m=+3741.043566136" watchObservedRunningTime="2026-07-28 11:25:21.770692455 +0000 UTC m=+3747.048629628"
Jul 28 11:25:23.642444 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:23.642414    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9whhf_f0f135df-5161-4f47-b431-8dca8b419610/manager/0.log"
Jul 28 11:25:24.036277 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:24.036253    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5k7t4_f8cda8cc-33ad-41e4-bd33-3a8f66243322/manager/0.log"
Jul 28 11:25:24.222407 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:24.222377    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5mh2r_a8c05a0a-bbec-46f7-bb87-aaa8685a5b77/server/0.log"
Jul 28 11:25:24.351691 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:24.351662    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zfj2q_9d0cc820-c243-41de-8bf8-d8b17b3c36f8/manager/0.log"
Jul 28 11:25:24.390674 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:24.390629    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f66f7dc-hb2m2_953e374c-964b-40a6-8ec5-3dd076748f39/manager/0.log"
Jul 28 11:25:24.408794 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:24.408763    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f66f7dc-hb2m2_953e374c-964b-40a6-8ec5-3dd076748f39/copy-manifests/0.log"
Jul 28 11:25:25.676891 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:25.676865    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-77785_035a1d4f-39a7-4f52-94c3-2ac37fdaed68/istio-proxy/0.log"
Jul 28 11:25:25.849854 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:25.849816    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d6b86846-rkl5w_ca029121-a68d-4531-9152-23bf1a84c263/kube-auth-proxy/0.log"
Jul 28 11:25:25.868460 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:25.868436    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/0.log"
Jul 28 11:25:25.923116 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:25.923042    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/1.log"
Jul 28 11:25:25.983535 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:25.983504    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d49bfb5-n7bxk_435525f2-d692-4697-b299-0a6484bbd281/router/0.log"
Jul 28 11:25:27.329998 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:27.329966    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-77785_035a1d4f-39a7-4f52-94c3-2ac37fdaed68/istio-proxy/0.log"
Jul 28 11:25:27.521271 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:27.521242    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d6b86846-rkl5w_ca029121-a68d-4531-9152-23bf1a84c263/kube-auth-proxy/0.log"
Jul 28 11:25:27.545824 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:27.545796    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/0.log"
Jul 28 11:25:27.600620 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:27.600596    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-628z8_37ec00fd-a31d-476d-baed-6719443b91fb/istio-proxy/1.log"
Jul 28 11:25:27.661483 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:27.661456    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d49bfb5-n7bxk_435525f2-d692-4697-b299-0a6484bbd281/router/0.log"
Jul 28 11:25:30.786348 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:30.786264    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-9hhr6_8da11606-2cc2-4ae0-82b1-5372c89aa961/migrator/0.log"
Jul 28 11:25:30.804475 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:30.804452    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-9hhr6_8da11606-2cc2-4ae0-82b1-5372c89aa961/graceful-termination/0.log"
Jul 28 11:25:31.148237 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:31.148209    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl_662b8125-a009-4bad-ba14-574df1776a3e/kube-storage-version-migrator-operator/1.log"
Jul 28 11:25:31.149947 ip-10-0-129-31 kubenswrapper[2544]: I0728 11:25:31.149919    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-wpmhl_662b8125-a009-4bad-ba14-574df1776a3e/kube-storage-version-migrator-operator/0.log"