Jul 30 16:36:10.620787 ip-10-0-132-133 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 16:36:11.026520 ip-10-0-132-133 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:11.026520 ip-10-0-132-133 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 16:36:11.026520 ip-10-0-132-133 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:11.026520 ip-10-0-132-133 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 16:36:11.026520 ip-10-0-132-133 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:11.028315 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.028197    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 16:36:11.031834 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031817    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.031834 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031831    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.031834 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031835    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031838    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031841    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031844    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031847    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031850    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031854    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031856    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031859    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031862    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031865    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031871    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031874    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031877    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031879    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031882    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031884    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031887    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031890    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031893    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.031926 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031896    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031898    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031901    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031903    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031906    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031909    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031911    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031914    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031916    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031919    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031921    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031924    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031927    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031930    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031933    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031936    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031938    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031941    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031944    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.032619 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031946    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031949    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031951    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031954    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.031956    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032009    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032055    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032060    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032064    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032068    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032073    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032077    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032085    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032089    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032094    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032099    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032104    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032109    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032120    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.033305 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032128    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032132    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032139    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032145    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032270    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032291    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032300    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032306    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032312    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032317    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032322    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032327    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032334    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032340    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032346    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032352    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032370    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032379    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.033822 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032384    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032389    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032394    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032399    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032404    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032409    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032413    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032418    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032423    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032427    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032436    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032442    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032446    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032451    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032456    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032461    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032466    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032471    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032476    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032500    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.034257 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.032506    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033955    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033967    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033971    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033974    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033977    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033979    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033982    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033986    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033988    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033991    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033994    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033996    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.033999    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034001    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034004    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034007    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034009    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034012    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034015    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.034751 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034017    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034020    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034023    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034025    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034028    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034030    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034034    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034036    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034039    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034042    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034044    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034046    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034049    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034052    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034055    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034057    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034060    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034062    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034065    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034068    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.035228 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034070    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034073    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034077    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034079    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034082    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034084    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034087    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034089    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034092    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034094    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034096    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034099    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034102    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034104    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034107    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034111    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034115    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034118    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034123    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.035720 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034127    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034130    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034132    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034135    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034138    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034141    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034145    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034148    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034150    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034153    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034156    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034159    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034161    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034164    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034167    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034169    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034172    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034174    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034177    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034179    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.036185 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034182    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034185    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034188    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034190    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034193    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034195    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034198    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034200    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034203    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034205    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034208    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034210    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034213    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034217    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034232    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034235    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034238    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034242    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034245    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.036682 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034249    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034251    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034659    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034665    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034669    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034672    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034674    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034677    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034680    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034683    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034685    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034688    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034690    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034693    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034695    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034698    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034701    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034703    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034706    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034708    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.037137 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034711    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034713    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034715    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034718    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034721    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034723    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034726    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034728    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034731    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034734    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034737    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034740    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034743    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034746    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034749    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034751    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034754    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034757    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034759    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.037640 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034762    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034764    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034767    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034769    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034772    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034774    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034777    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034779    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034782    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034784    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034787    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034789    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034791    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034794    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034796    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034799    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034801    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034803    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034806    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034808    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.038102 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034811    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034814    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034817    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034820    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034823    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034827    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034830    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034834    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034838    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034841    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034844    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034846    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034849    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034851    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034854    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034856    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034858    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034861    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034863    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.038653 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034866    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034868    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034871    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034873    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034876    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034878    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034881    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034883    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034885    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034888    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034890    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034893    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034895    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034898    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034901    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034904    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034906    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034910    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034914    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.039208 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034918    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034921    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034925    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.034929    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035002    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035017    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035023    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035027    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035031    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035034    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035038    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035043    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035046    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035049    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035052    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035056    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035059    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035062    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035065    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035069    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035072    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035075    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035085    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 30 16:36:11.039680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035088    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035091    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035094    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035098    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035106    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035109    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035112    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035116    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035119    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035122    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035126    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035129    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035132    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035136    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035139    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035142    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035145    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035148    2561 flags.go:64] FLAG: --enable-server="true"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035151    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035155    2561 flags.go:64] FLAG: --event-burst="100"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035158    2561 flags.go:64] FLAG: --event-qps="50"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035161    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035164    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035167    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 30 16:36:11.040226 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035171    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035174    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035177    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035183    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035186    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035189    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035192    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035196    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035199    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035202    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035205    2561 flags.go:64] FLAG: --feature-gates=""
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035209    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035212    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035215    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035218    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035221    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035224    2561 flags.go:64] FLAG: --help="false"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035227    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035230    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035234    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035237    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035241    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035244    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035247    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 16:36:11.040815 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035250    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035253    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035256    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035259    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035262    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035265    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035268    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035271    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035274    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035277    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035279    2561 flags.go:64] FLAG: --lock-file=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035284    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035287    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035290    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035295    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035298    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035301    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035304    2561 flags.go:64] FLAG: --logging-format="text"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035307    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035310    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035313    2561 flags.go:64] FLAG: --manifest-url=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035316    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035320    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035323    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035327    2561 flags.go:64] FLAG: --max-pods="110"
Jul 30 16:36:11.041419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035331    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035334    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035337    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035341    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035344    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035347    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035350    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035356    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035359    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035362    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035365    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035368    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035373    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035376    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035379    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035383    2561 flags.go:64] FLAG: --port="10250"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035386    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035389    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d2d48d822ae752ed"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035392    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035397    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035400    2561 flags.go:64] FLAG: --register-node="true"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035403    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035407    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035410    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 30 16:36:11.042074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035412    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035415    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035419    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035422    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035425    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035428    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035431    2561 flags.go:64] FLAG: --runonce="false"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035434    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035437    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035440    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035442    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035445    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035449    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035452    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035455    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035458    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035461    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035464    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035466    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035469    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035472    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035475    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035480    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035495    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035498    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 16:36:11.042673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035502    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035505    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035509    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035512    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035516    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035518    2561 flags.go:64] FLAG: --v="2"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035522    2561 flags.go:64] FLAG: --version="false"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035526    2561 flags.go:64] FLAG: --vmodule=""
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035530    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.035533    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035649    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035652    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035656    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035659    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035661    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035664    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035667    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035669    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035672    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035675    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035678    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035680    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.043266 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035683    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035685    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035688    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035690    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035693    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035695    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035698    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035701    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035703    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035705    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035708    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035711    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035714    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035717    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035720    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035723    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035726    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035729    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035731    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035734    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.043814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035737    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035739    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035742    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035744    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035747    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035749    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035752    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035754    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035757    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035761    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035765    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035768    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035771    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035773    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035776    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035779    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035781    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035784    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035786    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.044297 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035790    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035793    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035795    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035798    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035800    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035804    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035806    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035809    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035812    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035814    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035817    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035820    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035822    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035826    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035830    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035833    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035835    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035838    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035842    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.044766 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035846    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035849    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035851    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035854    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035857    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035859    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035862    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035865    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035868    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035871    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035873    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035876    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035878    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035882    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035885    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035887    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035890    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035893    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035896    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035899    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.045225 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035902    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035904    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035908    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035910    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035913    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035916    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035918    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035921    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.035923    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.036545    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.043339    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.043355    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.044208    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.044216    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044275    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.045724 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044280    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044284    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044287    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044290    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044293    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044295    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044298    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044301    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044304    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044307    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044309    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044312    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044315    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044318    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044320    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044323    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044325    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044328    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044331    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.046105 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044334    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044337    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044339    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044342    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044344    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044347    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044349    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044352    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044355    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044360    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044364    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044368    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044371    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044374    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044376    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044379    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044381    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044385    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044388    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044390    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.046586 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044393    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044396    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044399    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044402    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044404    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044407    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044409    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044412    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044414    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044417    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044419    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044422    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044425    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044428    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044431    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044435    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044438    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044441    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044443    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044446    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.047069 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044449    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044453    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044457    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044460    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044462    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044465    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044469    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044473    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044476    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044478    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044481    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044495    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044499    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044502    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044505    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044508    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044511    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044513    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044516    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044519    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.047597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044522    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044524    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044527    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044529    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044532    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044535    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044537    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044540    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044542    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044545    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044548    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044550    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044553    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044556    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044559    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044562    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044565    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044568    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.048081 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044570    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.044575    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044690    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044694    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044698    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044702    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044705    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044707    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044710    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044713    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044715    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044717    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044720    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044722    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044725    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:11.048534 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044728    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044730    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044732    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044735    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044737    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044740    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044742    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044745    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044747    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044750    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044752    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044754    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044757    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044760    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044763    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044766    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044769    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044772    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044775    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044777    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:11.048909 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044780    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044783    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044786    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044788    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044791    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044793    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044795    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044798    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044800    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044803    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044805    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044808    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044810    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044813    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044816    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044819    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044822    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044824    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044827    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044829    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:11.049392 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044832    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044834    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044837    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044839    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044842    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044845    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044847    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044850    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044853    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044856    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044858    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044861    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044863    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044866    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044868    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044871    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044873    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044875    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044878    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044880    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:11.050008 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044883    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044885    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044888    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044890    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044893    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044895    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044898    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044901    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044903    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044906    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044908    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044911    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044914    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044916    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044919    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044921    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044924    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044927    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044929    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044932    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:11.050527 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044934    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044939    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044942    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044945    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044948    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:11.044951    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.044956    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.045069    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.047027    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.048259    2561 server.go:1019] "Starting client certificate rotation"
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.048358    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.048391    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 16:36:11.051049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.050458    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 16:36:11.051602 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.051584    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 16:36:11.062542 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.062518    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 30 16:36:11.066663 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.066649    2561 log.go:25] "Validated CRI v1 image API"
Jul 30 16:36:11.067524 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.067507    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 16:36:11.070054 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.070036    2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9594f29d-3356-42f2-b663-68098803932e:/dev/nvme0n1p3 f5531fa6-53c5-49f5-af79-753c4fe89ce2:/dev/nvme0n1p4]
Jul 30 16:36:11.070120 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.070054    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 16:36:11.075971 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.075860    2561 manager.go:216] Machine: {Timestamp:2026-07-30 16:36:11.073760859 +0000 UTC m=+0.331222292 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092926 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a36de96749578b61bfcf96eb1f92 SystemUUID:ec24a36d-e967-4957-8b61-bfcf96eb1f92 BootID:c3b6ae57-4c91-4a5d-9129-56272fbbb46d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:bf:46:8e:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:bf:46:8e:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:90:27:7a:a9:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 16:36:11.075971 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.075968    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 16:36:11.076105 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.076093    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 16:36:11.079290 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.079263    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 16:36:11.079439 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.079293    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 16:36:11.080616 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.080596    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 16:36:11.080616 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.080619    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 16:36:11.080689 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.080627    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 16:36:11.081344 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.081333    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 16:36:11.082280 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.082270    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084081    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084202    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084232    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084244    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084259    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084268    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 30 16:36:11.084296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084279    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 16:36:11.084706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084691    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 16:36:11.084757 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.084717    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 16:36:11.087024 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.087003    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 16:36:11.089028 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.089014    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 16:36:11.089102 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.089046    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 16:36:11.090364 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.090341    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 16:36:11.091278 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091261    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091283    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091290    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091298    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091308    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091319    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091328    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091333    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091340    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 16:36:11.091343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091346    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 16:36:11.091677 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091361    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 16:36:11.091677 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.091374    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 16:36:11.092922 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.092904    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 16:36:11.092922 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.092921    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 16:36:11.096253 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.096238    2561 server.go:1268] "Started kubelet"
Jul 30 16:36:11.096385 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.096355    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 16:36:11.099731 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.096413    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 16:36:11.099731 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.099721    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 16:36:11.099914 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.099888    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 16:36:11.099914 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.099904    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 16:36:11.100063 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.100042    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 16:36:11.100722 ip-10-0-132-133 systemd[1]: Started Kubernetes Kubelet.
Jul 30 16:36:11.102432 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.101460    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 16:36:11.103807 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.103792    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 16:36:11.106603 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.106581    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 16:36:11.110732 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.110715    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 16:36:11.110732 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.110716    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 16:36:11.111079 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.111045    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.111588 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.111541    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 16:36:11.111970 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.111943    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 16:36:11.111970 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.111963    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 16:36:11.112131 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112070    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 16:36:11.112131 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112084    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 16:36:11.112889 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112868    2561 factory.go:153] Registering CRI-O factory
Jul 30 16:36:11.112889 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112891    2561 factory.go:223] Registration of the crio container factory successfully
Jul 30 16:36:11.113010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112953    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 16:36:11.113010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112964    2561 factory.go:55] Registering systemd factory
Jul 30 16:36:11.113010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112970    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 30 16:36:11.113010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.112990    2561 factory.go:103] Registering Raw factory
Jul 30 16:36:11.113010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.113002    2561 manager.go:1196] Started watching for new ooms in manager
Jul 30 16:36:11.113667 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.113649    2561 manager.go:349] Starting recovery of all containers
Jul 30 16:36:11.117916 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.117882    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 16:36:11.120069 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.117773    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-133.ec2.internal.18c71e8dabbd08fe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-133.ec2.internal,UID:ip-10-0-132-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-133.ec2.internal,},FirstTimestamp:2026-07-30 16:36:11.096205566 +0000 UTC m=+0.353666982,LastTimestamp:2026-07-30 16:36:11.096205566 +0000 UTC m=+0.353666982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-133.ec2.internal,}"
Jul 30 16:36:11.120069 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.120035    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 16:36:11.121119 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.121089    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vv8np"
Jul 30 16:36:11.124845 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.124823    2561 manager.go:354] Recovery completed
Jul 30 16:36:11.129448 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.129435    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.132094 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.132072    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.132205 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.132111    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vv8np"
Jul 30 16:36:11.132205 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.132124    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.132205 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.132145    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.133272 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.133252    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 16:36:11.133272 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.133271    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 16:36:11.133388 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.133296    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 16:36:11.134312 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.134290    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.135257 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.134894    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-133.ec2.internal.18c71e8dade0ab9f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-133.ec2.internal,UID:ip-10-0-132-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-133.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-133.ec2.internal,},FirstTimestamp:2026-07-30 16:36:11.132095391 +0000 UTC m=+0.389556812,LastTimestamp:2026-07-30 16:36:11.132095391 +0000 UTC m=+0.389556812,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-133.ec2.internal,}"
Jul 30 16:36:11.135384 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135350    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.135434 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135388    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.135434 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135404    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.135830 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135814    2561 policy_none.go:49] "None policy: Start"
Jul 30 16:36:11.135876 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135834    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 16:36:11.135876 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.135846    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 16:36:11.137336 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.137325    2561 policy_none.go:47] "Start"
Jul 30 16:36:11.171248 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171103    2561 manager.go:337] "Starting Device Plugin manager"
Jul 30 16:36:11.171318 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.171263    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 16:36:11.171318 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171276    2561 server.go:87] "Starting device plugin registration server"
Jul 30 16:36:11.171592 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171578    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 16:36:11.171661 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171597    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 16:36:11.171717 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171704    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 16:36:11.171804 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171783    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 16:36:11.171804 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.171799    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 16:36:11.172343 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.172319    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 16:36:11.172429 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.172381    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.275688 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.275649    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.277745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.277651    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.277745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.277690    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.277745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.277706    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.277745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.277739    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.284881 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.284864    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.284958 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.284889    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-133.ec2.internal\": node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.302999 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.302971    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.332888 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.332856    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 16:36:11.334136 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.334117    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 16:36:11.334248 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.334154    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 16:36:11.334248 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.334175    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 16:36:11.334248 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.334210    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 16:36:11.339162 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.339142    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:11.406577 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.406531    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.435717 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.435687    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"]
Jul 30 16:36:11.435807 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.435765    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.437016 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.436999    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.437083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.437035    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.437083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.437051    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.438482 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.438466    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.438606 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.438590    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.438666 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.438625    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.439396 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439375    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.439493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439407    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.439493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439427    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.439493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439437    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.439493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439452    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.439493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.439462    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.441028 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.441010    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.441121 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.441040    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:11.441894 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.441878    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:11.441974 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.441904    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:11.441974 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.441921    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:11.470382 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.470363    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-133.ec2.internal\" not found" node="ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.475108 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.475090    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-133.ec2.internal\" not found" node="ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.506708 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.506682    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.513200 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.513184    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.513293 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.513207    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.513293 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.513225    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ceb15f8e15e94f7eca74166940c49314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-133.ec2.internal\" (UID: \"ceb15f8e15e94f7eca74166940c49314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.606867 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.606825    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.614384 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614343    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ceb15f8e15e94f7eca74166940c49314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-133.ec2.internal\" (UID: \"ceb15f8e15e94f7eca74166940c49314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.614384 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ceb15f8e15e94f7eca74166940c49314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-133.ec2.internal\" (UID: \"ceb15f8e15e94f7eca74166940c49314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.614572 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614391    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.614572 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.614572 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.614572 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.614443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33966f0255bf1a59e2183879d0a73db9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal\" (UID: \"33966f0255bf1a59e2183879d0a73db9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.708546 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.708479    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.778307 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.778273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.778436 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.778313    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:11.809544 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.809514    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.916971 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:11.916885    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:11.927845 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:11.927814    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:12.017124 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.017092    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.049084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.049039    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 16:36:12.049593 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.049215    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 16:36:12.049593 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.049250    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 16:36:12.117756 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.117717    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.138162 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.138117    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 16:31:11 +0000 UTC" deadline="2028-04-26 07:18:38.954721648 +0000 UTC"
Jul 30 16:36:12.138162 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.138159    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15254h42m26.81656793s"
Jul 30 16:36:12.218725 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.218648    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.319187 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.319139    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.343025 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:12.342988    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33966f0255bf1a59e2183879d0a73db9.slice/crio-6463d7c7d16b70bc34f79325e30164ba156e69d412c9d0a0af8f0412870e934c WatchSource:0}: Error finding container 6463d7c7d16b70bc34f79325e30164ba156e69d412c9d0a0af8f0412870e934c: Status 404 returned error can't find the container with id 6463d7c7d16b70bc34f79325e30164ba156e69d412c9d0a0af8f0412870e934c
Jul 30 16:36:12.347247 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.347230    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:36:12.402754 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:12.402725    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb15f8e15e94f7eca74166940c49314.slice/crio-1db10d07854802e35f5b130f26c852fb11e84de16beef3ac83165125cdfe85c4 WatchSource:0}: Error finding container 1db10d07854802e35f5b130f26c852fb11e84de16beef3ac83165125cdfe85c4: Status 404 returned error can't find the container with id 1db10d07854802e35f5b130f26c852fb11e84de16beef3ac83165125cdfe85c4
Jul 30 16:36:12.420032 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.420005    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.492780 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.492686    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:12.520103 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:12.520064    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-133.ec2.internal\" not found"
Jul 30 16:36:12.552280 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.552250    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:12.612536 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.612499    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:12.624770 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.624741    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 16:36:12.625844 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.625831    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal"
Jul 30 16:36:12.632983 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:12.632968    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 16:36:13.086730 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.086694    2561 apiserver.go:52] "Watching apiserver"
Jul 30 16:36:13.095069 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.095041    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 16:36:13.096121 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.096073    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-m5bt4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal","openshift-multus/multus-additional-cni-plugins-jr6nf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg","openshift-multus/multus-7c2fx","openshift-multus/network-metrics-daemon-8lzgx","openshift-network-diagnostics/network-check-target-87vzb","openshift-network-operator/iptables-alerter-9r99m","openshift-ovn-kubernetes/ovnkube-node-qrnv9","kube-system/global-pull-secret-syncer-mkxgs","kube-system/konnectivity-agent-r6x55","kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal","openshift-cluster-node-tuning-operator/tuned-khpgg","openshift-dns/node-resolver-mdp4w"]
Jul 30 16:36:13.097971 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.097918    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:13.098080 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.098005    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:13.099297 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.099274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.100690 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.100666    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.102002 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.101978    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.102843 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.102822    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jd2l2\""
Jul 30 16:36:13.103101 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.103058    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.103834 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.103815    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hpbmd\""
Jul 30 16:36:13.104144 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.104127    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 16:36:13.104335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.104319    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.104496 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.104479    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 16:36:13.104806 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.104788    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.105035 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.105019    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.105526 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.105504    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.105630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.105604    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.105686 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.105659    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:13.105932 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.105912    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 16:36:13.106408 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.106390    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.108033 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108010    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.108228 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108213    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 16:36:13.108430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108412    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mlhp2\""
Jul 30 16:36:13.108430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108427    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.108656 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108637    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 16:36:13.108656 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108651    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.108850 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.108831    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zwlwg\""
Jul 30 16:36:13.109411 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.109388    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.109615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.109597    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 16:36:13.109722 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.109634    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.110289 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110268    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kwgrs\""
Jul 30 16:36:13.110387 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110298    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.110495 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110479    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 16:36:13.110495 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110490    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 16:36:13.110704 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110631    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 16:36:13.110847 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110812    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 16:36:13.111476 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.110980    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.111476 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.111156    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 16:36:13.111672 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.111531    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.111672 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.111613    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4c6mb\""
Jul 30 16:36:13.114846 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.113679    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.114846 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.113938    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.114846 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.114520    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.114846 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.114719    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:13.115528 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.115141    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m9skf\""
Jul 30 16:36:13.115648 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.115570    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.115648 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.115595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.118205 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.118183    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 16:36:13.119114 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119063    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 16:36:13.119265 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119123    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:13.119649 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119630    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xnsf6\""
Jul 30 16:36:13.119824 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119806    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9wkgx\""
Jul 30 16:36:13.119923 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119747    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:13.120222 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.119674    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 16:36:13.123809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.123790    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 16:36:13.124084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124064    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.124216 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-run\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.124334 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-kubelet\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.124452 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124345    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/177be080-0a05-4ad1-bb8e-6b162983cf44-host\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.124452 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124372    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-device-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.124452 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124396    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.124452 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124432    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124456    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-cni-binary-copy\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124480    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-netns\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-netns\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124527    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/177be080-0a05-4ad1-bb8e-6b162983cf44-serviceca\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124577    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfj9q\" (UniqueName: \"kubernetes.io/projected/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-kube-api-access-wfj9q\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124690    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-multus\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124715    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-systemd-units\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-var-lib-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.124762 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124766    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-bin\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124790    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-script-lib\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124831    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkz45\" (UniqueName: \"kubernetes.io/projected/f214bfb0-038b-4596-8951-c85042453a54-kube-api-access-dkz45\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124855    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-lib-modules\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124893    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124922    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-system-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-bin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-etc-kubernetes\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.124996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-socket-dir-parent\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125048    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-sys-fs\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b541c7e8-e31d-449c-b433-6d366cebdd63-iptables-alerter-script\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-cnibin\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-cnibin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125146    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-config\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-socket-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.125466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-multus-daemon-config\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125209    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-slash\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125229    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f214bfb0-038b-4596-8951-c85042453a54-hosts-file\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125253    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a71b369-4cc8-4eaa-aa69-bced7def4dce-agent-certs\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125315    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-multus-certs\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-node-log\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w7cm\" (UniqueName: \"kubernetes.io/projected/de4fa743-0ef2-470f-b389-8eb9516cca1d-kube-api-access-9w7cm\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125387    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f214bfb0-038b-4596-8951-c85042453a54-tmp-dir\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125412    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125436    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a71b369-4cc8-4eaa-aa69-bced7def4dce-konnectivity-ca\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-etc-tuned\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125500    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-hostroot\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-conf\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125580    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-etc-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhmkz\" (UniqueName: \"kubernetes.io/projected/b541c7e8-e31d-449c-b433-6d366cebdd63-kube-api-access-lhmkz\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125623    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysconfig\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.126444 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-kubernetes\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-conf-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125694    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovn-node-metrics-cert\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-registration-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125748    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-etc-selinux\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125779    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j288c\" (UniqueName: \"kubernetes.io/projected/786c5433-6d72-48c5-802b-c4c20bbd7842-kube-api-access-j288c\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-modprobe-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125829    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-sys\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125851    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-tmp\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125902    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvml7\" (UniqueName: \"kubernetes.io/projected/c187172d-eb83-49a3-aac1-f79ae469809e-kube-api-access-zvml7\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125928    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-env-overrides\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-kubelet-config\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.125977    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126017    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-systemd\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126043    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-os-release\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-kubelet\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.127490 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-ovn\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126154    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-dbus\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-var-lib-kubelet\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126243    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xkvw\" (UniqueName: \"kubernetes.io/projected/d6179fe3-8140-43e4-a8f8-944a69858eaf-kube-api-access-9xkvw\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126269    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126300    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vwxk\" (UniqueName: \"kubernetes.io/projected/7c0f597f-7f78-4106-a184-57e07649743d-kube-api-access-8vwxk\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126322    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-systemd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-log-socket\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126389    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzfjr\" (UniqueName: \"kubernetes.io/projected/177be080-0a05-4ad1-bb8e-6b162983cf44-kube-api-access-bzfjr\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126423    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b541c7e8-e31d-449c-b433-6d366cebdd63-host-slash\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-host\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126473    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-os-release\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126519    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-k8s-cni-cncf-io\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.128335 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.126568    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-netd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.151046 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.149166    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4pghx"
Jul 30 16:36:13.158888 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.158639    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4pghx"
Jul 30 16:36:13.213237 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.213213    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 16:36:13.227453 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227424    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b541c7e8-e31d-449c-b433-6d366cebdd63-iptables-alerter-script\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.227604 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-cnibin\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.227604 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-cnibin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.227604 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-config\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.227604 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-socket-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.227604 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-multus-daemon-config\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227651    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-cnibin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227733    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-slash\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227636    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-cnibin\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-slash\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f214bfb0-038b-4596-8951-c85042453a54-hosts-file\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.227893 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227803    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-socket-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.228191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a71b369-4cc8-4eaa-aa69-bced7def4dce-agent-certs\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.228191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.227971    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f214bfb0-038b-4596-8951-c85042453a54-hosts-file\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.228191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228019    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.228191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228061    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-multus-certs\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.228191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228131    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-node-log\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228196    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w7cm\" (UniqueName: \"kubernetes.io/projected/de4fa743-0ef2-470f-b389-8eb9516cca1d-kube-api-access-9w7cm\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228209    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b541c7e8-e31d-449c-b433-6d366cebdd63-iptables-alerter-script\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228227    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f214bfb0-038b-4596-8951-c85042453a54-tmp-dir\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228249    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-multus-certs\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228249    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228198    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-node-log\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228325    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-multus-daemon-config\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228385    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a71b369-4cc8-4eaa-aa69-bced7def4dce-konnectivity-ca\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.228461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-etc-tuned\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-hostroot\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-conf\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228570    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-etc-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhmkz\" (UniqueName: \"kubernetes.io/projected/b541c7e8-e31d-449c-b433-6d366cebdd63-kube-api-access-lhmkz\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysconfig\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228643    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f214bfb0-038b-4596-8951-c85042453a54-tmp-dir\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-kubernetes\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-conf-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228724    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovn-node-metrics-cert\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228749    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-registration-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228838    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-registration-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228858    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-config\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228900    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysconfig\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.228979    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-kubernetes\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-hostroot\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229037    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-sysctl-conf\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229079    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-etc-selinux\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j288c\" (UniqueName: \"kubernetes.io/projected/786c5433-6d72-48c5-802b-c4c20bbd7842-kube-api-access-j288c\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-modprobe-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229081    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-etc-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229161    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-sys\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-tmp\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvml7\" (UniqueName: \"kubernetes.io/projected/c187172d-eb83-49a3-aac1-f79ae469809e-kube-api-access-zvml7\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229241    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-env-overrides\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229272    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a71b369-4cc8-4eaa-aa69-bced7def4dce-konnectivity-ca\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-kubelet-config\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229335    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-kubelet-config\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229357    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-systemd\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229385    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-os-release\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.229693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229395    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-sys\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229416    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-kubelet\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229476    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-kubelet\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229709    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-systemd\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229688    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-etc-selinux\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-ovn\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229811    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-dbus\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-var-lib-kubelet\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xkvw\" (UniqueName: \"kubernetes.io/projected/d6179fe3-8140-43e4-a8f8-944a69858eaf-kube-api-access-9xkvw\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229849    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-conf-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229965    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230023    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-ovn\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229355    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-etc-modprobe-d\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.230576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230094    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741ecd44-3b6f-4429-9900-e03e8d8710b9-dbus\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.229852    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-os-release\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230159    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-var-lib-kubelet\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.229159    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-env-overrides\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.230313    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vwxk\" (UniqueName: \"kubernetes.io/projected/7c0f597f-7f78-4106-a184-57e07649743d-kube-api-access-8vwxk\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.231083    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:13.731054668 +0000 UTC m=+2.988516085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231318    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-systemd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.231391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-log-socket\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231406    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzfjr\" (UniqueName: \"kubernetes.io/projected/177be080-0a05-4ad1-bb8e-6b162983cf44-kube-api-access-bzfjr\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b541c7e8-e31d-449c-b433-6d366cebdd63-host-slash\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231479    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-host\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231507    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-os-release\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-k8s-cni-cncf-io\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.231786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231739    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-systemd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.232151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231794    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-log-socket\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.232151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231842    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.232151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.231999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b541c7e8-e31d-449c-b433-6d366cebdd63-host-slash\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.232151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232058    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-host\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.232151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232119    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-os-release\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.232380 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232160    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-k8s-cni-cncf-io\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.232380 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232271    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-netd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.232380 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232305    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.232380 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-run\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.232380 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232339    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-netd\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232402    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-run\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232452    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-kubelet\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232475    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-etc-tuned\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/177be080-0a05-4ad1-bb8e-6b162983cf44-host\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232540    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-device-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232575    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.232630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-kubelet\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-cni-binary-copy\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/177be080-0a05-4ad1-bb8e-6b162983cf44-host\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-netns\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-netns\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232871    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/177be080-0a05-4ad1-bb8e-6b162983cf44-serviceca\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-run-netns\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.232988    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfj9q\" (UniqueName: \"kubernetes.io/projected/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-kube-api-access-wfj9q\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.233019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233052    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-multus\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233082    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-systemd-units\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-var-lib-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-bin\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-script-lib\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233211    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkz45\" (UniqueName: \"kubernetes.io/projected/f214bfb0-038b-4596-8951-c85042453a54-kube-api-access-dkz45\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233249    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c0f597f-7f78-4106-a184-57e07649743d-cni-binary-copy\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-lib-modules\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233330    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-device-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-system-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-bin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.233386    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-etc-kubernetes\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.233485    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:13.733465793 +0000 UTC m=+2.990927208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:13.233493 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233486    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-system-cni-dir\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233510    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-run-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233539    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-socket-dir-parent\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-run-netns\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233596    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-sys-fs\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233599    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-etc-kubernetes\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233732    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-multus-socket-dir-parent\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233783    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-sys-fs\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233790    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-var-lib-openvswitch\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233846    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-multus\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233901    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-systemd-units\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233942    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6179fe3-8140-43e4-a8f8-944a69858eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/786c5433-6d72-48c5-802b-c4c20bbd7842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.233980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/177be080-0a05-4ad1-bb8e-6b162983cf44-serviceca\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.234118    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c187172d-eb83-49a3-aac1-f79ae469809e-lib-modules\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.234282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.234228    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c0f597f-7f78-4106-a184-57e07649743d-host-var-lib-cni-bin\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.235058 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.234514    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de4fa743-0ef2-470f-b389-8eb9516cca1d-host-cni-bin\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.235058 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.234561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovnkube-script-lib\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.235058 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.234652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6179fe3-8140-43e4-a8f8-944a69858eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.236895 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.236150    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:13.236895 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.236175    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:13.236895 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.236195    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:13.236895 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.236256    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:13.736234728 +0000 UTC m=+2.993696128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:13.236895 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.236348    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de4fa743-0ef2-470f-b389-8eb9516cca1d-ovn-node-metrics-cert\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.237523 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.237395    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a71b369-4cc8-4eaa-aa69-bced7def4dce-agent-certs\") pod \"konnectivity-agent-r6x55\" (UID: \"1a71b369-4cc8-4eaa-aa69-bced7def4dce\") " pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.239390 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.239291    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c187172d-eb83-49a3-aac1-f79ae469809e-tmp\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.239390 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.239335    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhmkz\" (UniqueName: \"kubernetes.io/projected/b541c7e8-e31d-449c-b433-6d366cebdd63-kube-api-access-lhmkz\") pod \"iptables-alerter-9r99m\" (UID: \"b541c7e8-e31d-449c-b433-6d366cebdd63\") " pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.239991 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.239969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j288c\" (UniqueName: \"kubernetes.io/projected/786c5433-6d72-48c5-802b-c4c20bbd7842-kube-api-access-j288c\") pod \"aws-ebs-csi-driver-node-4b9rg\" (UID: \"786c5433-6d72-48c5-802b-c4c20bbd7842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.240524 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.240497    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvml7\" (UniqueName: \"kubernetes.io/projected/c187172d-eb83-49a3-aac1-f79ae469809e-kube-api-access-zvml7\") pod \"tuned-khpgg\" (UID: \"c187172d-eb83-49a3-aac1-f79ae469809e\") " pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.241302 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.241276    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xkvw\" (UniqueName: \"kubernetes.io/projected/d6179fe3-8140-43e4-a8f8-944a69858eaf-kube-api-access-9xkvw\") pod \"multus-additional-cni-plugins-jr6nf\" (UID: \"d6179fe3-8140-43e4-a8f8-944a69858eaf\") " pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.242067 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.242011    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzfjr\" (UniqueName: \"kubernetes.io/projected/177be080-0a05-4ad1-bb8e-6b162983cf44-kube-api-access-bzfjr\") pod \"node-ca-m5bt4\" (UID: \"177be080-0a05-4ad1-bb8e-6b162983cf44\") " pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.242650 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.242625    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w7cm\" (UniqueName: \"kubernetes.io/projected/de4fa743-0ef2-470f-b389-8eb9516cca1d-kube-api-access-9w7cm\") pod \"ovnkube-node-qrnv9\" (UID: \"de4fa743-0ef2-470f-b389-8eb9516cca1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.243669 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.243645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkz45\" (UniqueName: \"kubernetes.io/projected/f214bfb0-038b-4596-8951-c85042453a54-kube-api-access-dkz45\") pod \"node-resolver-mdp4w\" (UID: \"f214bfb0-038b-4596-8951-c85042453a54\") " pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.243767 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.243645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vwxk\" (UniqueName: \"kubernetes.io/projected/7c0f597f-7f78-4106-a184-57e07649743d-kube-api-access-8vwxk\") pod \"multus-7c2fx\" (UID: \"7c0f597f-7f78-4106-a184-57e07649743d\") " pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.244225 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.244205    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfj9q\" (UniqueName: \"kubernetes.io/projected/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-kube-api-access-wfj9q\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.321157 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.321123    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:13.339466 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.339361    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal" event={"ID":"ceb15f8e15e94f7eca74166940c49314","Type":"ContainerStarted","Data":"1db10d07854802e35f5b130f26c852fb11e84de16beef3ac83165125cdfe85c4"}
Jul 30 16:36:13.340651 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.340613    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal" event={"ID":"33966f0255bf1a59e2183879d0a73db9","Type":"ContainerStarted","Data":"6463d7c7d16b70bc34f79325e30164ba156e69d412c9d0a0af8f0412870e934c"}
Jul 30 16:36:13.419589 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.419553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-khpgg"
Jul 30 16:36:13.431764 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.431741    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jr6nf"
Jul 30 16:36:13.442756 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.442724    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg"
Jul 30 16:36:13.451710 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.451678    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7c2fx"
Jul 30 16:36:13.461664 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.461637    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:13.469567 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.469542    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m5bt4"
Jul 30 16:36:13.478567 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.478534    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mdp4w"
Jul 30 16:36:13.488281 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.488257    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9r99m"
Jul 30 16:36:13.495137 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.495116    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:13.737241 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.737152    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:13.737241 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.737215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:13.737277    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737332    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737393    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737410    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:14.737391113 +0000 UTC m=+3.994852516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737415    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737435    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:13.737451 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737445    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:13.737734 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737445    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:14.737429388 +0000 UTC m=+3.994890794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:13.737734 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:13.737512    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:14.737482066 +0000 UTC m=+3.994943469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:14.161840 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.161805    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 16:31:13 +0000 UTC" deadline="2028-01-11 03:58:23.706740972 +0000 UTC"
Jul 30 16:36:14.161840 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.161833    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12707h22m9.544910825s"
Jul 30 16:36:14.264245 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.264217    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf214bfb0_038b_4596_8951_c85042453a54.slice/crio-6912ab563674df70c5752a3ae2206858754a2faddb156f38913c3ffd06428ddd WatchSource:0}: Error finding container 6912ab563674df70c5752a3ae2206858754a2faddb156f38913c3ffd06428ddd: Status 404 returned error can't find the container with id 6912ab563674df70c5752a3ae2206858754a2faddb156f38913c3ffd06428ddd
Jul 30 16:36:14.281833 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.281801    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc187172d_eb83_49a3_aac1_f79ae469809e.slice/crio-baeb8720004de1f432ce707f558154902a9d1db659c6997d476099e954fd39cc WatchSource:0}: Error finding container baeb8720004de1f432ce707f558154902a9d1db659c6997d476099e954fd39cc: Status 404 returned error can't find the container with id baeb8720004de1f432ce707f558154902a9d1db659c6997d476099e954fd39cc
Jul 30 16:36:14.283718 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.283695    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0f597f_7f78_4106_a184_57e07649743d.slice/crio-04579db4778fa789f8d9e844e79bc0aaad623a3fde1be0ec989033267d17e1ec WatchSource:0}: Error finding container 04579db4778fa789f8d9e844e79bc0aaad623a3fde1be0ec989033267d17e1ec: Status 404 returned error can't find the container with id 04579db4778fa789f8d9e844e79bc0aaad623a3fde1be0ec989033267d17e1ec
Jul 30 16:36:14.284671 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.284643    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177be080_0a05_4ad1_bb8e_6b162983cf44.slice/crio-2da1170b2fc1602c2df40e9308bce27e4fef823f2c5c1e7d8b66cbbcd9967c90 WatchSource:0}: Error finding container 2da1170b2fc1602c2df40e9308bce27e4fef823f2c5c1e7d8b66cbbcd9967c90: Status 404 returned error can't find the container with id 2da1170b2fc1602c2df40e9308bce27e4fef823f2c5c1e7d8b66cbbcd9967c90
Jul 30 16:36:14.285407 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.285350    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a71b369_4cc8_4eaa_aa69_bced7def4dce.slice/crio-475f70b09ce9c8c6dfe869bb7d8fe033bdf82b88ca6bcf23d7eacda725074aa9 WatchSource:0}: Error finding container 475f70b09ce9c8c6dfe869bb7d8fe033bdf82b88ca6bcf23d7eacda725074aa9: Status 404 returned error can't find the container with id 475f70b09ce9c8c6dfe869bb7d8fe033bdf82b88ca6bcf23d7eacda725074aa9
Jul 30 16:36:14.286420 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.286366    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb541c7e8_e31d_449c_b433_6d366cebdd63.slice/crio-6c22688fb725e4477175a3baa317813cee5979294360c06ffc053ca463681a4f WatchSource:0}: Error finding container 6c22688fb725e4477175a3baa317813cee5979294360c06ffc053ca463681a4f: Status 404 returned error can't find the container with id 6c22688fb725e4477175a3baa317813cee5979294360c06ffc053ca463681a4f
Jul 30 16:36:14.287460 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.287435    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4fa743_0ef2_470f_b389_8eb9516cca1d.slice/crio-8c5e2d3998237d444b01958d3f983ee951a8e9809b2be70d0cc867598af6c41f WatchSource:0}: Error finding container 8c5e2d3998237d444b01958d3f983ee951a8e9809b2be70d0cc867598af6c41f: Status 404 returned error can't find the container with id 8c5e2d3998237d444b01958d3f983ee951a8e9809b2be70d0cc867598af6c41f
Jul 30 16:36:14.288402 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.288294    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6179fe3_8140_43e4_a8f8_944a69858eaf.slice/crio-6c19356da4b7577605dca439d110eb9aeba38c47064540ecbcbdb9fd632b207a WatchSource:0}: Error finding container 6c19356da4b7577605dca439d110eb9aeba38c47064540ecbcbdb9fd632b207a: Status 404 returned error can't find the container with id 6c19356da4b7577605dca439d110eb9aeba38c47064540ecbcbdb9fd632b207a
Jul 30 16:36:14.289664 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:14.289644    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786c5433_6d72_48c5_802b_c4c20bbd7842.slice/crio-4ee3cbe612fae44357da039f393ff1803aed027c41231c37f243ff50bdf68a4d WatchSource:0}: Error finding container 4ee3cbe612fae44357da039f393ff1803aed027c41231c37f243ff50bdf68a4d: Status 404 returned error can't find the container with id 4ee3cbe612fae44357da039f393ff1803aed027c41231c37f243ff50bdf68a4d
Jul 30 16:36:14.343311 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.343136    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-khpgg" event={"ID":"c187172d-eb83-49a3-aac1-f79ae469809e","Type":"ContainerStarted","Data":"baeb8720004de1f432ce707f558154902a9d1db659c6997d476099e954fd39cc"}
Jul 30 16:36:14.344362 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.344307    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mdp4w" event={"ID":"f214bfb0-038b-4596-8951-c85042453a54","Type":"ContainerStarted","Data":"6912ab563674df70c5752a3ae2206858754a2faddb156f38913c3ffd06428ddd"}
Jul 30 16:36:14.345461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.345438    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg" event={"ID":"786c5433-6d72-48c5-802b-c4c20bbd7842","Type":"ContainerStarted","Data":"4ee3cbe612fae44357da039f393ff1803aed027c41231c37f243ff50bdf68a4d"}
Jul 30 16:36:14.346640 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.346613    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerStarted","Data":"6c19356da4b7577605dca439d110eb9aeba38c47064540ecbcbdb9fd632b207a"}
Jul 30 16:36:14.347831 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.347807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"8c5e2d3998237d444b01958d3f983ee951a8e9809b2be70d0cc867598af6c41f"}
Jul 30 16:36:14.348982 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.348936    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m5bt4" event={"ID":"177be080-0a05-4ad1-bb8e-6b162983cf44","Type":"ContainerStarted","Data":"2da1170b2fc1602c2df40e9308bce27e4fef823f2c5c1e7d8b66cbbcd9967c90"}
Jul 30 16:36:14.350426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.350403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9r99m" event={"ID":"b541c7e8-e31d-449c-b433-6d366cebdd63","Type":"ContainerStarted","Data":"6c22688fb725e4477175a3baa317813cee5979294360c06ffc053ca463681a4f"}
Jul 30 16:36:14.351603 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.351464    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6x55" event={"ID":"1a71b369-4cc8-4eaa-aa69-bced7def4dce","Type":"ContainerStarted","Data":"475f70b09ce9c8c6dfe869bb7d8fe033bdf82b88ca6bcf23d7eacda725074aa9"}
Jul 30 16:36:14.355056 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.355030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7c2fx" event={"ID":"7c0f597f-7f78-4106-a184-57e07649743d","Type":"ContainerStarted","Data":"04579db4778fa789f8d9e844e79bc0aaad623a3fde1be0ec989033267d17e1ec"}
Jul 30 16:36:14.397851 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.397824    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:14.744753 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.744636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:14.744753 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.744731    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:14.744971 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:14.744780    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:14.744971 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.744924    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:14.745076 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.744988    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.744970581 +0000 UTC m=+6.002431983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:14.745566 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745407    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:14.745566 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745464    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.74544725 +0000 UTC m=+6.002908650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:14.745761 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745594    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:14.745761 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745617    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:14.745761 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745631    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:14.745761 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:14.745686    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.745670585 +0000 UTC m=+6.003131982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:15.162460 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.162362    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 16:31:13 +0000 UTC" deadline="2028-04-02 03:17:26.052328952 +0000 UTC"
Jul 30 16:36:15.162460 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.162425    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14674h41m10.889908307s"
Jul 30 16:36:15.336376 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.335606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:15.336376 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:15.335752    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:15.336376 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.336136    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:15.336376 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:15.336254    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:15.337112 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.336859    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:15.337112 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:15.336952    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:15.377016 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.376979    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal" event={"ID":"ceb15f8e15e94f7eca74166940c49314","Type":"ContainerStarted","Data":"7cde22f1291114f459e03647200ec882d7ff919cd1d117729d736eb4fa154d88"}
Jul 30 16:36:15.391870 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.391833    2561 generic.go:356] "Generic (PLEG): container finished" podID="33966f0255bf1a59e2183879d0a73db9" containerID="12c823c60fc5b60f8e9d7beaace374bb8cfc484ea2b66fc8b5b52a11530af66a" exitCode=0
Jul 30 16:36:15.391870 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.391881    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal" event={"ID":"33966f0255bf1a59e2183879d0a73db9","Type":"ContainerDied","Data":"12c823c60fc5b60f8e9d7beaace374bb8cfc484ea2b66fc8b5b52a11530af66a"}
Jul 30 16:36:15.393144 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:15.393094    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-133.ec2.internal" podStartSLOduration=3.393076443 podStartE2EDuration="3.393076443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:15.392321201 +0000 UTC m=+4.649782624" watchObservedRunningTime="2026-07-30 16:36:15.393076443 +0000 UTC m=+4.650537864"
Jul 30 16:36:16.418751 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:16.418710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal" event={"ID":"33966f0255bf1a59e2183879d0a73db9","Type":"ContainerStarted","Data":"ecced1509f983b0cc84e0bf78076cdce511576c606d74f4ce5adbd8e2f1861b6"}
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:16.762912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:16.762985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:16.763023    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763171    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763191    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763205    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763276    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:20.763254179 +0000 UTC m=+10.020715595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763344    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763380    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:20.763368137 +0000 UTC m=+10.020829561 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763436    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:16.763857 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:16.763465    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:20.763454794 +0000 UTC m=+10.020916245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:17.334988 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:17.334956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:17.335175 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:17.335087    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:17.335520 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:17.335473    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:17.335622 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:17.335603    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:17.335679 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:17.335652    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:17.335734 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:17.335720    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:19.335431 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:19.335383    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:19.335892 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:19.335396    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:19.335892 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:19.335560    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:19.335892 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:19.335617    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:19.335892 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:19.335746    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:19.335892 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:19.335840    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:20.795616    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:20.795677    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:20.795724    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.795862    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.795925    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:28.79590369 +0000 UTC m=+18.053365111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796193    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796214    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796227    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796273    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:28.796258737 +0000 UTC m=+18.053720150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796294    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:20.796353 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:20.796333    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:28.796321143 +0000 UTC m=+18.053782546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:21.335770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:21.335898    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:21.335930    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:21.336026    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:21.336047    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:21.336250 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:21.336101    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:23.335075 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:23.335039    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:23.335582 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:23.335042    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:23.335582 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:23.335177    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:23.335582 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:23.335255    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:23.335582 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:23.335039    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:23.335582 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:23.335327    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:25.335334 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:25.335290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:25.335334 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:25.335329    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:25.335825 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:25.335294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:25.335825 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:25.335420    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:25.335825 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:25.335518    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:25.335825 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:25.335587    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:27.335403 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:27.335322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:27.335873 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:27.335334    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:27.335873 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:27.335467    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:27.335873 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:27.335559    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:27.335873 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:27.335626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:27.335873 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:27.335713    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:28.853796 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:28.853761    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:28.853806    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:28.853846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.853937    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.853954    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.853975    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.853991    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.854006    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.854019    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:44.853997917 +0000 UTC m=+34.111459327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.854038    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:44.854028769 +0000 UTC m=+34.111490171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:28.854316 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:28.854057    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:44.85404745 +0000 UTC m=+34.111508860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:29.334810 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:29.334776    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:29.334985 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:29.334776    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:29.334985 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:29.334909    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:29.335109 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:29.334783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:29.335109 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:29.334985    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:29.335109 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:29.335085    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:31.335998 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:31.335953    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:31.336462 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:31.336061    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:31.336462 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:31.336090    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:31.336462 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:31.336057    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:31.336462 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:31.336175    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:31.336462 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:31.336297    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:33.334898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.334657    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:33.335553 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.334658    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:33.335553 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.334658    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:33.335553 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:33.335021    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:33.335553 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:33.335118    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:33.335553 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:33.335229    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:33.451460 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.451425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg" event={"ID":"786c5433-6d72-48c5-802b-c4c20bbd7842","Type":"ContainerStarted","Data":"1ed1688a13c39c362c0b3866ef87f5ffcea56eba645b131a9896e2e31185af8a"}
Jul 30 16:36:33.452824 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.452798    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="72b60738e5dc99fbf11342855ea24bc1c67a0d5186cdd64ccf72dab08a5949b6" exitCode=0
Jul 30 16:36:33.452953 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.452871    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"72b60738e5dc99fbf11342855ea24bc1c67a0d5186cdd64ccf72dab08a5949b6"}
Jul 30 16:36:33.455772 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.455754    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:36:33.456175 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456106    2561 generic.go:356] "Generic (PLEG): container finished" podID="de4fa743-0ef2-470f-b389-8eb9516cca1d" containerID="a388733ba93a7b27e7ec1f40f6f1f62896622f2457be4b690fad3aa9689e62c6" exitCode=1
Jul 30 16:36:33.456229 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456173    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"c8b0fa2f978ff81931a5efe9891888af5674f2d936b8dedfd963c753010f1a78"}
Jul 30 16:36:33.456294 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"3301232b8616211d599d63e5a9c132c9aff2faca0ba6bfe07a3946d9985bceee"}
Jul 30 16:36:33.456343 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456306    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"bb32ee5efb188df202efe72642da929c2ef8b0efb4f40f66a7fa4c924f49867f"}
Jul 30 16:36:33.456396 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"76b79fbbc86fdf43c2856e07ab0601642d669a9119765217d972d3a58f26d023"}
Jul 30 16:36:33.456445 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456408    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerDied","Data":"a388733ba93a7b27e7ec1f40f6f1f62896622f2457be4b690fad3aa9689e62c6"}
Jul 30 16:36:33.456518 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.456457    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"8379c4ba34f5bc27f0cf39ee18203afa9b1d4cd20941357e9e5608a1d96c4263"}
Jul 30 16:36:33.459439 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.459409    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m5bt4" event={"ID":"177be080-0a05-4ad1-bb8e-6b162983cf44","Type":"ContainerStarted","Data":"a684f2832859d670eb09f8aa2408916ae76720e86f95ceb5fb7609c27f5faba3"}
Jul 30 16:36:33.460789 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.460759    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6x55" event={"ID":"1a71b369-4cc8-4eaa-aa69-bced7def4dce","Type":"ContainerStarted","Data":"3ffb426e08ed548834e5ec3621a4e6b82a36ed4d1f84b55d34c8357207551bf4"}
Jul 30 16:36:33.462222 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.462200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7c2fx" event={"ID":"7c0f597f-7f78-4106-a184-57e07649743d","Type":"ContainerStarted","Data":"a0cba23b376fe95d96d5685a92f79fd92867df39e76b930e714a5b5f52e02365"}
Jul 30 16:36:33.463630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.463608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-khpgg" event={"ID":"c187172d-eb83-49a3-aac1-f79ae469809e","Type":"ContainerStarted","Data":"dee97528b95dc74eb928af36392cbcf0a414e9a17082308e1af541c6e0d91e3c"}
Jul 30 16:36:33.464926 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.464910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mdp4w" event={"ID":"f214bfb0-038b-4596-8951-c85042453a54","Type":"ContainerStarted","Data":"738a51a5e7738fd4ab3911908ff01c8a2d3640a06ad7a2daccf9828041bd5c04"}
Jul 30 16:36:33.489089 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.489054    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-133.ec2.internal" podStartSLOduration=21.489043131 podStartE2EDuration="21.489043131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:16.459383903 +0000 UTC m=+5.716845348" watchObservedRunningTime="2026-07-30 16:36:33.489043131 +0000 UTC m=+22.746504551"
Jul 30 16:36:33.498356 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.498324    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7c2fx" podStartSLOduration=3.772278292 podStartE2EDuration="22.498313616s" totalImagesPullingTime="18.726035324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:33.497914935 +0000 UTC m=+22.755376354" watchObservedRunningTime="2026-07-30 16:36:33.498313616 +0000 UTC m=+22.755775038"
Jul 30 16:36:33.512759 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.512726    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m5bt4" podStartSLOduration=4.084110326 podStartE2EDuration="22.512714792s" totalImagesPullingTime="18.428604466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:33.512519225 +0000 UTC m=+22.769980674" watchObservedRunningTime="2026-07-30 16:36:33.512714792 +0000 UTC m=+22.770176211"
Jul 30 16:36:33.532569 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.530036    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-khpgg" podStartSLOduration=4.094332682 podStartE2EDuration="22.530025932s" totalImagesPullingTime="18.43569325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:33.529946889 +0000 UTC m=+22.787408299" watchObservedRunningTime="2026-07-30 16:36:33.530025932 +0000 UTC m=+22.787487351"
Jul 30 16:36:33.544827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.544786    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mdp4w" podStartSLOduration=4.10451683 podStartE2EDuration="22.544772114s" totalImagesPullingTime="18.440255284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:33.544286017 +0000 UTC m=+22.801747440" watchObservedRunningTime="2026-07-30 16:36:33.544772114 +0000 UTC m=+22.802233534"
Jul 30 16:36:33.558523 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:33.558473    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r6x55" podStartSLOduration=8.54137085 podStartE2EDuration="22.558464707s" totalImagesPullingTime="14.017093857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:33.558179062 +0000 UTC m=+22.815640481" watchObservedRunningTime="2026-07-30 16:36:33.558464707 +0000 UTC m=+22.815926182"
Jul 30 16:36:34.469019 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:34.468942    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9r99m" event={"ID":"b541c7e8-e31d-449c-b433-6d366cebdd63","Type":"ContainerStarted","Data":"bed5f457732df2075a5733a0aa75dee858001d35eb9c5b7ffaccb64ed0ccaf53"}
Jul 30 16:36:34.485576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:34.485448    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9r99m" podStartSLOduration=5.058567237 podStartE2EDuration="23.485417259s" totalImagesPullingTime="18.426850022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:34.485057271 +0000 UTC m=+23.742518694" watchObservedRunningTime="2026-07-30 16:36:34.485417259 +0000 UTC m=+23.742878678"
Jul 30 16:36:34.653782 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:34.653749    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 16:36:35.185127 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.184943    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T16:36:34.653777322Z","UUID":"1d92a36a-f06e-4be6-9571-14488374776c","Handler":null,"Name":"","Endpoint":""}
Jul 30 16:36:35.186701 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.186669    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 16:36:35.186827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.186711    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 16:36:35.335402 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.335032    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:35.335402 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:35.335167    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:35.335402 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.335271    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:35.335402 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:35.335403    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:35.335732 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.335440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:35.335732 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:35.335524    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:35.473317 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:35.473227    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg" event={"ID":"786c5433-6d72-48c5-802b-c4c20bbd7842","Type":"ContainerStarted","Data":"358f499058f7fd7ec2890a1d8d9873011358af84cc2b3e517a8cd7cc47079fb0"}
Jul 30 16:36:36.479369 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:36.479340    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:36:36.479840 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:36.479786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"4a8557021fb2c7ac9c708d00770ddc13398a5744ab704e2107d199febd177c12"}
Jul 30 16:36:36.484301 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:36.484238    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg" event={"ID":"786c5433-6d72-48c5-802b-c4c20bbd7842","Type":"ContainerStarted","Data":"b6ea185db50a74d9b2e42f8daf5f0ddd17c6c24665fb6dd225f717fe487ce57e"}
Jul 30 16:36:36.508561 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:36.508520    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4b9rg" podStartSLOduration=4.312697997 podStartE2EDuration="25.508505075s" totalImagesPullingTime="21.195807078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:36.508084763 +0000 UTC m=+25.765546204" watchObservedRunningTime="2026-07-30 16:36:36.508505075 +0000 UTC m=+25.765966492"
Jul 30 16:36:37.334719 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:37.334531    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:37.334890 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:37.334531    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:37.334890 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:37.334829    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:37.334890 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:37.334873    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:37.334890 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:37.334597    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:37.335104 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:37.334958    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:38.326453 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.326277    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:38.327112 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.326824    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:38.490432 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.490399    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="dea11952c4673cf948f845309a915472a4bbebff8db41f1e60ddceefc61cfe30" exitCode=0
Jul 30 16:36:38.490614 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.490479    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"dea11952c4673cf948f845309a915472a4bbebff8db41f1e60ddceefc61cfe30"}
Jul 30 16:36:38.493772 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.493746    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:36:38.495620 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"391a05d810b9990e92a02da05a54609b21bf53a0d919dffa7833e5940dad35ac"}
Jul 30 16:36:38.495620 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495517    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:38.495620 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495545    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:38.495620 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495599    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:38.496447 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495647    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:38.496447 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495544    2561 scope.go:117] "RemoveContainer" containerID="a388733ba93a7b27e7ec1f40f6f1f62896622f2457be4b690fad3aa9689e62c6"
Jul 30 16:36:38.496447 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.495716    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r6x55"
Jul 30 16:36:38.514333 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.514314    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:38.514550 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:38.514536    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:36:39.335188 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.335154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:39.335188 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.335154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:39.335692 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.335154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:39.335692 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.335294    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:39.335692 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.335407    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:39.335692 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.335530    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:39.500337 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.500311    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:36:39.500756 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.500727    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" event={"ID":"de4fa743-0ef2-470f-b389-8eb9516cca1d","Type":"ContainerStarted","Data":"ce9823b322487bd167433d7e0a925c111fa8ca6b5f18b311fd18c1e7ae27641b"}
Jul 30 16:36:39.531143 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.531095    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9" podStartSLOduration=10.064568435 podStartE2EDuration="28.531080013s" totalImagesPullingTime="18.466511578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:39.530541105 +0000 UTC m=+28.788002527" watchObservedRunningTime="2026-07-30 16:36:39.531080013 +0000 UTC m=+28.788541432"
Jul 30 16:36:39.691797 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.691635    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8lzgx"]
Jul 30 16:36:39.691932 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.691896    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:39.692035 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.692015    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:39.699920 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.699897    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mkxgs"]
Jul 30 16:36:39.700029 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.699977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:39.700089 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.700072    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:39.722297 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.722274    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-87vzb"]
Jul 30 16:36:39.722395 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:39.722346    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:39.722433 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:39.722415    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:40.504548 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:40.504515    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="45c74b758975db849a98509ed68029be501654cd4dfbcfc7278c17204e8661b2" exitCode=0
Jul 30 16:36:40.504942 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:40.504612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"45c74b758975db849a98509ed68029be501654cd4dfbcfc7278c17204e8661b2"}
Jul 30 16:36:41.335764 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:41.335736    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:41.335926 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:41.335832    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:41.335926 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:41.335844    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:41.335926 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:41.335862    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:41.336075 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:41.335963    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:41.336075 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:41.336041    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:42.514024 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:42.513992    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="de13160e471749bc790047307242e4816057eb5a94f383827966e2fcc3bd5189" exitCode=0
Jul 30 16:36:42.514768 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:42.514038    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"de13160e471749bc790047307242e4816057eb5a94f383827966e2fcc3bd5189"}
Jul 30 16:36:43.337584 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:43.337556    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:43.337737 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:43.337650    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:43.337737 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:43.337660    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:43.337737 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:43.337656    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:43.337831 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:43.337733    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:43.337831 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:43.337789    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:44.872408 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:44.872372    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:44.872447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:44.872502    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872543    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872615    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret podName:741ecd44-3b6f-4429-9900-e03e8d8710b9 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:16.872594181 +0000 UTC m=+66.130055594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret") pod "global-pull-secret-syncer-mkxgs" (UID: "741ecd44-3b6f-4429-9900-e03e8d8710b9") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872624    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872644    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872669    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872682    2561 projected.go:196] Error preparing data for projected volume kube-api-access-mcbln for pod openshift-network-diagnostics/network-check-target-87vzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872682    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs podName:54a8a9b3-f13d-4b3a-9729-9bb7b1252705 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:16.872664925 +0000 UTC m=+66.130126323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs") pod "network-metrics-daemon-8lzgx" (UID: "54a8a9b3-f13d-4b3a-9729-9bb7b1252705") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:44.872956 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:44.872737    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln podName:3dbcfb66-0274-4978-b49e-76b2eeb342c0 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:16.872722942 +0000 UTC m=+66.130184343 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcbln" (UniqueName: "kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln") pod "network-check-target-87vzb" (UID: "3dbcfb66-0274-4978-b49e-76b2eeb342c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:45.335208 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:45.335175    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:45.335208 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:45.335208    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:45.335425 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:45.335174    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:45.335425 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:45.335312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8lzgx" podUID="54a8a9b3-f13d-4b3a-9729-9bb7b1252705"
Jul 30 16:36:45.335532 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:45.335461    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkxgs" podUID="741ecd44-3b6f-4429-9900-e03e8d8710b9"
Jul 30 16:36:45.335607 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:45.335586    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-87vzb" podUID="3dbcfb66-0274-4978-b49e-76b2eeb342c0"
Jul 30 16:36:46.564109 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.564084    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-133.ec2.internal" event="NodeReady"
Jul 30 16:36:46.564688 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.564222    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 16:36:46.618479 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.618442    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nl5h6"]
Jul 30 16:36:46.621036 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.621015    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-99jng"]
Jul 30 16:36:46.621164 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.621136    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.622939 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.622919    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.623388 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.623362    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xkdnd\""
Jul 30 16:36:46.623388 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.623380    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 16:36:46.623553 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.623426    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 16:36:46.623553 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.623479    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 16:36:46.625007 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.624990    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 16:36:46.625150 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.625131    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 16:36:46.625230 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.625214    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59qbk\""
Jul 30 16:36:46.636469 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.636047    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99jng"]
Jul 30 16:36:46.639598 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.639569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nl5h6"]
Jul 30 16:36:46.787252 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dckm\" (UniqueName: \"kubernetes.io/projected/bc28418d-5fec-4aac-a992-e57d9832495b-kube-api-access-9dckm\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.787252 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc28418d-5fec-4aac-a992-e57d9832495b-config-volume\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.787252 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787246    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx9dj\" (UniqueName: \"kubernetes.io/projected/86204387-75d5-43b6-be45-9f6b12167f39-kube-api-access-mx9dj\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.787562 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787264    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.787562 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787300    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86204387-75d5-43b6-be45-9f6b12167f39-cert\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.787562 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.787349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc28418d-5fec-4aac-a992-e57d9832495b-tmp-dir\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.888091 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888054    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx9dj\" (UniqueName: \"kubernetes.io/projected/86204387-75d5-43b6-be45-9f6b12167f39-kube-api-access-mx9dj\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.888091 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888093    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.888310 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86204387-75d5-43b6-be45-9f6b12167f39-cert\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.888310 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc28418d-5fec-4aac-a992-e57d9832495b-tmp-dir\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.888411 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:46.888374    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:46.888475 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dckm\" (UniqueName: \"kubernetes.io/projected/bc28418d-5fec-4aac-a992-e57d9832495b-kube-api-access-9dckm\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.888475 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:46.888461    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:36:47.388438668 +0000 UTC m=+36.645900085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:46.888615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc28418d-5fec-4aac-a992-e57d9832495b-tmp-dir\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.888615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.888529    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc28418d-5fec-4aac-a992-e57d9832495b-config-volume\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.889135 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.889110    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc28418d-5fec-4aac-a992-e57d9832495b-config-volume\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.893018 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.892995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86204387-75d5-43b6-be45-9f6b12167f39-cert\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.898202 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.898175    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dckm\" (UniqueName: \"kubernetes.io/projected/bc28418d-5fec-4aac-a992-e57d9832495b-kube-api-access-9dckm\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:46.898662 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.898636    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx9dj\" (UniqueName: \"kubernetes.io/projected/86204387-75d5-43b6-be45-9f6b12167f39-kube-api-access-mx9dj\") pod \"ingress-canary-nl5h6\" (UID: \"86204387-75d5-43b6-be45-9f6b12167f39\") " pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:46.933387 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:46.933359    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nl5h6"
Jul 30 16:36:47.161396 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:47.161183    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86204387_75d5_43b6_be45_9f6b12167f39.slice/crio-f5b21261ec72b47119a2e3a2ff234335787345888f47f297a10227711d6041b4 WatchSource:0}: Error finding container f5b21261ec72b47119a2e3a2ff234335787345888f47f297a10227711d6041b4: Status 404 returned error can't find the container with id f5b21261ec72b47119a2e3a2ff234335787345888f47f297a10227711d6041b4
Jul 30 16:36:47.161396 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.161345    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nl5h6"]
Jul 30 16:36:47.334564 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.334538    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:36:47.334702 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.334581    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:36:47.334805 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.334788    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:36:47.342520 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.342461    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 16:36:47.342520 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.342493    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 16:36:47.343183 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.343158    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 16:36:47.343183 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.343174    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2m4bh\""
Jul 30 16:36:47.343337 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.343184    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 16:36:47.343337 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.343201    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2ksq\""
Jul 30 16:36:47.391730 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.391707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:47.391832 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:47.391822    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:47.391881 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:47.391867    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:36:48.391854516 +0000 UTC m=+37.649315918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:47.528198 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.528170    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="103fb9ae5bb41bbdc7bc05f6ac234bc0167d55e0482283dc002694165550b041" exitCode=0
Jul 30 16:36:47.528369 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.528261    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"103fb9ae5bb41bbdc7bc05f6ac234bc0167d55e0482283dc002694165550b041"}
Jul 30 16:36:47.529504 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:47.529383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nl5h6" event={"ID":"86204387-75d5-43b6-be45-9f6b12167f39","Type":"ContainerStarted","Data":"f5b21261ec72b47119a2e3a2ff234335787345888f47f297a10227711d6041b4"}
Jul 30 16:36:48.400318 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:48.400288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:48.400792 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:48.400416    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:48.400792 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:48.400517    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:36:50.400469338 +0000 UTC m=+39.657930736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:48.535631 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:48.535596    2561 generic.go:356] "Generic (PLEG): container finished" podID="d6179fe3-8140-43e4-a8f8-944a69858eaf" containerID="4a7e97c52aa22459fbd48ffeb3c0cd91a62580d19c8089c8f442ef0c04c88e70" exitCode=0
Jul 30 16:36:48.535795 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:48.535664    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerDied","Data":"4a7e97c52aa22459fbd48ffeb3c0cd91a62580d19c8089c8f442ef0c04c88e70"}
Jul 30 16:36:49.543229 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:49.542919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" event={"ID":"d6179fe3-8140-43e4-a8f8-944a69858eaf","Type":"ContainerStarted","Data":"c93f0cbfa3163948c17361787ba6056870b1e172b566986eddea0374fe67e277"}
Jul 30 16:36:49.567730 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:49.567679    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jr6nf" podStartSLOduration=9.415520134 podStartE2EDuration="38.56766295s" totalImagesPullingTime="29.152142816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:49.566238075 +0000 UTC m=+38.823699526" watchObservedRunningTime="2026-07-30 16:36:49.56766295 +0000 UTC m=+38.825124374"
Jul 30 16:36:50.416790 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:50.416756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:50.416941 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:50.416908    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:50.416980 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:50.416973    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:36:54.416958623 +0000 UTC m=+43.674420020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:50.547262 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:50.547228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nl5h6" event={"ID":"86204387-75d5-43b6-be45-9f6b12167f39","Type":"ContainerStarted","Data":"4fbbc68b3fb09549ad150ffb7b363525718568b9fb134fa8158775914e620730"}
Jul 30 16:36:50.565390 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:50.565339    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nl5h6" podStartSLOduration=1.754169103 podStartE2EDuration="4.565323749s" totalImagesPullingTime="2.811154646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:50.564555879 +0000 UTC m=+39.822017314" watchObservedRunningTime="2026-07-30 16:36:50.565323749 +0000 UTC m=+39.822785162"
Jul 30 16:36:53.307042 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.307013    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"]
Jul 30 16:36:53.339476 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.339450    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"]
Jul 30 16:36:53.339636 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.339585    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"
Jul 30 16:36:53.342329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.342306    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:53.342426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.342415    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:53.343214 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.343197    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cng9z\""
Jul 30 16:36:53.438682 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.438651    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4np7\" (UniqueName: \"kubernetes.io/projected/9b77609b-cf37-4900-a46a-14f76238bf45-kube-api-access-p4np7\") pod \"volume-data-source-validator-549c6d54cf-xhz55\" (UID: \"9b77609b-cf37-4900-a46a-14f76238bf45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"
Jul 30 16:36:53.508890 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.508859    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"]
Jul 30 16:36:53.529251 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.529228    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6748bdbfdc-jrch2"]
Jul 30 16:36:53.529368 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.529354    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.531973 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.531920    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 30 16:36:53.532615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.532158    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:53.532615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.532259    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:53.532615 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.532260    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 30 16:36:53.533469 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.533451    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cck78\""
Jul 30 16:36:53.539557 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.539539    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4np7\" (UniqueName: \"kubernetes.io/projected/9b77609b-cf37-4900-a46a-14f76238bf45-kube-api-access-p4np7\") pod \"volume-data-source-validator-549c6d54cf-xhz55\" (UID: \"9b77609b-cf37-4900-a46a-14f76238bf45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"
Jul 30 16:36:53.553411 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.553385    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kskrz"]
Jul 30 16:36:53.553547 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.553529    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.555584 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.555564    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 30 16:36:53.555697 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.555686    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 16:36:53.555827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.555812    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 30 16:36:53.555827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.555822    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 16:36:53.555991 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.555893    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 30 16:36:53.556076 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.556062    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-47bt9\""
Jul 30 16:36:53.556143 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.556119    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 30 16:36:53.556497 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.556466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4np7\" (UniqueName: \"kubernetes.io/projected/9b77609b-cf37-4900-a46a-14f76238bf45-kube-api-access-p4np7\") pod \"volume-data-source-validator-549c6d54cf-xhz55\" (UID: \"9b77609b-cf37-4900-a46a-14f76238bf45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"
Jul 30 16:36:53.577558 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.577532    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"]
Jul 30 16:36:53.577652 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.577565    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6748bdbfdc-jrch2"]
Jul 30 16:36:53.577652 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.577581    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kskrz"]
Jul 30 16:36:53.577714 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.577658    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.579818 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.579798    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 16:36:53.580194 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.580175    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:53.580279 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.580202    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-46s65\""
Jul 30 16:36:53.580326 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.580284    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 16:36:53.580518 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.580480    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:53.586793 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.586777    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 16:36:53.640358 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.640335    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m66n\" (UniqueName: \"kubernetes.io/projected/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-kube-api-access-2m66n\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.640458 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.640377    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.640458 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.640402    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.648541 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.648525    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"
Jul 30 16:36:53.740832 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.740804    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.740932 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.740861    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-config\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.740932 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.740895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.740932 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.740924    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.741091 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.740948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-serving-cert\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.741091 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-stats-auth\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741102    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-trusted-ca\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741159    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m66n\" (UniqueName: \"kubernetes.io/projected/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-kube-api-access-2m66n\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741532    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741553    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741586    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-default-certificate\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741622    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtgqm\" (UniqueName: \"kubernetes.io/projected/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-kube-api-access-vtgqm\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.742156 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.741659    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2mcf\" (UniqueName: \"kubernetes.io/projected/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-kube-api-access-r2mcf\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.743504 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.743469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.751159 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.751137    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m66n\" (UniqueName: \"kubernetes.io/projected/22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09-kube-api-access-2m66n\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c\" (UID: \"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.789291 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.789261    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55"]
Jul 30 16:36:53.792994 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:53.792966    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b77609b_cf37_4900_a46a_14f76238bf45.slice/crio-967be94e5526ad6c3f5201acd87fdc2f56f607181be714b7734ad6896a74e1fa WatchSource:0}: Error finding container 967be94e5526ad6c3f5201acd87fdc2f56f607181be714b7734ad6896a74e1fa: Status 404 returned error can't find the container with id 967be94e5526ad6c3f5201acd87fdc2f56f607181be714b7734ad6896a74e1fa
Jul 30 16:36:53.839589 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.839568    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"
Jul 30 16:36:53.842844 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.842826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-config\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.842902 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.842854    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.842974 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.842956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-serving-cert\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.842974 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:53.842970    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:54.342955849 +0000 UTC m=+43.600417251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:36:53.843083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843012    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-stats-auth\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.843083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843027    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-trusted-ca\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.843183 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843134    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.843183 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-default-certificate\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.843183 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtgqm\" (UniqueName: \"kubernetes.io/projected/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-kube-api-access-vtgqm\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.843344 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843190    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2mcf\" (UniqueName: \"kubernetes.io/projected/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-kube-api-access-r2mcf\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.843344 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:53.843263    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:36:53.843344 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:53.843335    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:54.343316059 +0000 UTC m=+43.600777459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:36:53.843644 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.843622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-config\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.844146 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.844126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-trusted-ca\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.845279 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.845257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-serving-cert\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.845384 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.845367    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-stats-auth\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.845426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.845411    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-default-certificate\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.853289 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.853271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2mcf\" (UniqueName: \"kubernetes.io/projected/9f5c0566-e219-43bd-8ecf-de5a4e7eed78-kube-api-access-r2mcf\") pod \"console-operator-584dcb6b54-kskrz\" (UID: \"9f5c0566-e219-43bd-8ecf-de5a4e7eed78\") " pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:53.853581 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.853565    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtgqm\" (UniqueName: \"kubernetes.io/projected/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-kube-api-access-vtgqm\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:53.886802 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:53.886783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:36:54.021141 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:54.021114    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b967ae_4cc3_4cff_bcd0_3a0e9ccc6a09.slice/crio-6a409e6b6ade0efaf926344c99c71d6e355629fe8fa4de1f9b17d82a1ff92d44 WatchSource:0}: Error finding container 6a409e6b6ade0efaf926344c99c71d6e355629fe8fa4de1f9b17d82a1ff92d44: Status 404 returned error can't find the container with id 6a409e6b6ade0efaf926344c99c71d6e355629fe8fa4de1f9b17d82a1ff92d44
Jul 30 16:36:54.027856 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.027830    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c"]
Jul 30 16:36:54.056682 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.056654    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kskrz"]
Jul 30 16:36:54.059149 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:36:54.059125    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5c0566_e219_43bd_8ecf_de5a4e7eed78.slice/crio-d7cea7caf6149aabd7b16310310332c6c8e5b90b0ce5e4f2483cc0ff380f5dcd WatchSource:0}: Error finding container d7cea7caf6149aabd7b16310310332c6c8e5b90b0ce5e4f2483cc0ff380f5dcd: Status 404 returned error can't find the container with id d7cea7caf6149aabd7b16310310332c6c8e5b90b0ce5e4f2483cc0ff380f5dcd
Jul 30 16:36:54.348044 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.348014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:54.348377 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.348082    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:54.348377 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:54.348162    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:36:54.348377 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:54.348183    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:55.348165972 +0000 UTC m=+44.605627372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:36:54.348377 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:54.348206    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:55.348198608 +0000 UTC m=+44.605660006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:36:54.448842 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.448812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:36:54.448987 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:54.448958    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:54.449022 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:54.449016    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:37:02.448998689 +0000 UTC m=+51.706460087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:54.557191 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.557159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" event={"ID":"9f5c0566-e219-43bd-8ecf-de5a4e7eed78","Type":"ContainerStarted","Data":"d7cea7caf6149aabd7b16310310332c6c8e5b90b0ce5e4f2483cc0ff380f5dcd"}
Jul 30 16:36:54.558282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.558261    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c" event={"ID":"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09","Type":"ContainerStarted","Data":"6a409e6b6ade0efaf926344c99c71d6e355629fe8fa4de1f9b17d82a1ff92d44"}
Jul 30 16:36:54.559402 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:54.559377    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55" event={"ID":"9b77609b-cf37-4900-a46a-14f76238bf45","Type":"ContainerStarted","Data":"967be94e5526ad6c3f5201acd87fdc2f56f607181be714b7734ad6896a74e1fa"}
Jul 30 16:36:55.355699 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:55.355663    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:55.356148 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:55.355740    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:55.356148 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:55.355837    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:36:55.356148 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:55.355908    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:57.355889039 +0000 UTC m=+46.613350437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:36:55.356148 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:55.355932    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:57.355920651 +0000 UTC m=+46.613382052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:36:57.372702 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.372675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:57.373048 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.372736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:36:57.373048 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:57.372833    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:36:57.373048 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:57.372843    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:01.372829011 +0000 UTC m=+50.630290409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:36:57.373048 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:57.372906    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:01.372888191 +0000 UTC m=+50.630349600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:36:57.569319 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.569235    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55" event={"ID":"9b77609b-cf37-4900-a46a-14f76238bf45","Type":"ContainerStarted","Data":"ed21b66950e5353bc05b9e0463a7942c81f9ad884d0df0805932d0f2ca5c2896"}
Jul 30 16:36:57.570690 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.570668    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/0.log"
Jul 30 16:36:57.570823 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.570701    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f5c0566-e219-43bd-8ecf-de5a4e7eed78" containerID="439269496e44956139aea6ac6660a9211c9fc39b6c1f179e2582fcb5efba34ac" exitCode=255
Jul 30 16:36:57.570823 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.570769    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" event={"ID":"9f5c0566-e219-43bd-8ecf-de5a4e7eed78","Type":"ContainerDied","Data":"439269496e44956139aea6ac6660a9211c9fc39b6c1f179e2582fcb5efba34ac"}
Jul 30 16:36:57.570943 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.570926    2561 scope.go:117] "RemoveContainer" containerID="439269496e44956139aea6ac6660a9211c9fc39b6c1f179e2582fcb5efba34ac"
Jul 30 16:36:57.572195 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.572175    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c" event={"ID":"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09","Type":"ContainerStarted","Data":"c0041b7b1d6913c81cfcd7641a2a2a811ae024a0c9fcdce8342dccc277f19dd0"}
Jul 30 16:36:57.585840 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.585800    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xhz55" podStartSLOduration=1.1670221060000001 podStartE2EDuration="4.585787062s" totalImagesPullingTime="3.418764956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:57.584935224 +0000 UTC m=+46.842396671" watchObservedRunningTime="2026-07-30 16:36:57.585787062 +0000 UTC m=+46.843248489"
Jul 30 16:36:57.634298 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:57.634255    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c" podStartSLOduration=1.414674974 podStartE2EDuration="4.634240788s" totalImagesPullingTime="3.219565814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:57.633199592 +0000 UTC m=+46.890661012" watchObservedRunningTime="2026-07-30 16:36:57.634240788 +0000 UTC m=+46.891702207"
Jul 30 16:36:58.576352 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.576283    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:36:58.576769 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.576740    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/0.log"
Jul 30 16:36:58.576813 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.576773    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f5c0566-e219-43bd-8ecf-de5a4e7eed78" containerID="87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1" exitCode=255
Jul 30 16:36:58.576861 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.576845    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" event={"ID":"9f5c0566-e219-43bd-8ecf-de5a4e7eed78","Type":"ContainerDied","Data":"87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1"}
Jul 30 16:36:58.576898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.576892    2561 scope.go:117] "RemoveContainer" containerID="439269496e44956139aea6ac6660a9211c9fc39b6c1f179e2582fcb5efba34ac"
Jul 30 16:36:58.577103 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.577085    2561 scope.go:117] "RemoveContainer" containerID="87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1"
Jul 30 16:36:58.577298 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:58.577271    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-kskrz_openshift-console-operator(9f5c0566-e219-43bd-8ecf-de5a4e7eed78)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" podUID="9f5c0566-e219-43bd-8ecf-de5a4e7eed78"
Jul 30 16:36:58.848399 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.848368    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-djcbt"]
Jul 30 16:36:58.879929 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.879904    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-djcbt"]
Jul 30 16:36:58.880073 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.880003    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:58.883041 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.883022    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 16:36:58.883041 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.883032    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 16:36:58.883360 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.883345    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vspgg\""
Jul 30 16:36:58.883431 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.883375    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 16:36:58.883899 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.883884    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 16:36:58.984523 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.984478    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:58.984706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.984547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:58.984706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.984641    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/862368d2-77e5-425b-a497-1caf9e2cb930-crio-socket\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:58.984706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.984680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/862368d2-77e5-425b-a497-1caf9e2cb930-data-volume\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:58.984843 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:58.984728    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx79l\" (UniqueName: \"kubernetes.io/projected/862368d2-77e5-425b-a497-1caf9e2cb930-kube-api-access-bx79l\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.085759 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.085732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx79l\" (UniqueName: \"kubernetes.io/projected/862368d2-77e5-425b-a497-1caf9e2cb930-kube-api-access-bx79l\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.085885 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.085784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.085885 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.085821    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.085885 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.085856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/862368d2-77e5-425b-a497-1caf9e2cb930-crio-socket\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.085885 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.085874    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/862368d2-77e5-425b-a497-1caf9e2cb930-data-volume\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.086142 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.085912    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.086142 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.085967    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.086142 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.085978    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:59.5859595 +0000 UTC m=+48.843420898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.086142 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.086024    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:59.586007841 +0000 UTC m=+48.843469239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.086142 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.086064    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/862368d2-77e5-425b-a497-1caf9e2cb930-crio-socket\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.086313 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.086142    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/862368d2-77e5-425b-a497-1caf9e2cb930-data-volume\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.096481 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.096454    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx79l\" (UniqueName: \"kubernetes.io/projected/862368d2-77e5-425b-a497-1caf9e2cb930-kube-api-access-bx79l\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.580676 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.580649    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:36:59.581031 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.580954    2561 scope.go:117] "RemoveContainer" containerID="87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1"
Jul 30 16:36:59.581102 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.581083    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-kskrz_openshift-console-operator(9f5c0566-e219-43bd-8ecf-de5a4e7eed78)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" podUID="9f5c0566-e219-43bd-8ecf-de5a4e7eed78"
Jul 30 16:36:59.588454 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.588436    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.588554 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:36:59.588522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:36:59.588593 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.588567    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.588593 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.588577    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.588661 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.588610    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:00.588596097 +0000 UTC m=+49.846057494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.588661 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:36:59.588624    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:00.588617471 +0000 UTC m=+49.846078869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : configmap "kube-rbac-proxy" not found
Jul 30 16:37:00.545375 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.545343    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p8kcp"]
Jul 30 16:37:00.555017 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.554997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.556247 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.556221    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p8kcp"]
Jul 30 16:37:00.558647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.558615    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 30 16:37:00.558647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.558640    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 30 16:37:00.559625 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.559608    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 30 16:37:00.559725 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.559658    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 30 16:37:00.559725 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.559692    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bvlfn\""
Jul 30 16:37:00.596480 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.596456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:00.596859 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.596534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:00.596859 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:00.596587    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 16:37:00.596859 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:00.596635    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:00.596859 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:00.596656    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:02.596636184 +0000 UTC m=+51.854097583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : configmap "kube-rbac-proxy" not found
Jul 30 16:37:00.596859 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:00.596674    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:02.596663028 +0000 UTC m=+51.854124426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:00.696851 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.696825    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-key\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.696978 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.696857    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77rnh\" (UniqueName: \"kubernetes.io/projected/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-kube-api-access-77rnh\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.696978 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.696934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-cabundle\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.797549 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.797456    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-key\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.797549 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.797506    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77rnh\" (UniqueName: \"kubernetes.io/projected/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-kube-api-access-77rnh\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.797758 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.797586    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-cabundle\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.798207 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.798190    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-cabundle\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.799931 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.799909    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-signing-key\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.806319 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.806297    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77rnh\" (UniqueName: \"kubernetes.io/projected/25a6606a-f4ea-4711-a29c-40c44cdd8c6d-kube-api-access-77rnh\") pod \"service-ca-658d7b7c78-p8kcp\" (UID: \"25a6606a-f4ea-4711-a29c-40c44cdd8c6d\") " pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.865544 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.865522    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp"
Jul 30 16:37:00.996886 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:00.996855    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p8kcp"]
Jul 30 16:37:01.001579 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:01.001552    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a6606a_f4ea_4711_a29c_40c44cdd8c6d.slice/crio-62529bfd8fe9875bf37ba17549055f3e84c7cdcb722b5ebfc67a57285a37c892 WatchSource:0}: Error finding container 62529bfd8fe9875bf37ba17549055f3e84c7cdcb722b5ebfc67a57285a37c892: Status 404 returned error can't find the container with id 62529bfd8fe9875bf37ba17549055f3e84c7cdcb722b5ebfc67a57285a37c892
Jul 30 16:37:01.402270 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:01.402232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:01.402442 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:01.402361    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:01.402442 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:01.402377    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:37:01.402442 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:01.402436    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:09.402421193 +0000 UTC m=+58.659882591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:37:01.402574 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:01.402481    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:09.402468576 +0000 UTC m=+58.659929974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:37:01.586302 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:01.586271    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp" event={"ID":"25a6606a-f4ea-4711-a29c-40c44cdd8c6d","Type":"ContainerStarted","Data":"62529bfd8fe9875bf37ba17549055f3e84c7cdcb722b5ebfc67a57285a37c892"}
Jul 30 16:37:01.644877 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:01.644853    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mdp4w_f214bfb0-038b-4596-8951-c85042453a54/dns-node-resolver/0.log"
Jul 30 16:37:02.513433 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:02.513398    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:02.513610 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:02.513560    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:37:02.513664 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:02.513623    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls podName:bc28418d-5fec-4aac-a992-e57d9832495b nodeName:}" failed. No retries permitted until 2026-07-30 16:37:18.513607913 +0000 UTC m=+67.771069311 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls") pod "dns-default-99jng" (UID: "bc28418d-5fec-4aac-a992-e57d9832495b") : secret "dns-default-metrics-tls" not found
Jul 30 16:37:02.614785 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:02.614758    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:02.614954 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:02.614814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:02.614954 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:02.614923    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:02.615049 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:02.614990    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:06.614969445 +0000 UTC m=+55.872430862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:02.615370 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:02.615348    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/862368d2-77e5-425b-a497-1caf9e2cb930-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:02.645124 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:02.645098    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m5bt4_177be080-0a05-4ad1-bb8e-6b162983cf44/node-ca/0.log"
Jul 30 16:37:03.445314 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.445292    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nl5h6_86204387-75d5-43b6-be45-9f6b12167f39/serve-healthcheck-canary/0.log"
Jul 30 16:37:03.593941 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.593906    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp" event={"ID":"25a6606a-f4ea-4711-a29c-40c44cdd8c6d","Type":"ContainerStarted","Data":"93ee63ac2673ce8936299fa97e58a45fd05ea70a51fa5a7585c62fa0ec3345fc"}
Jul 30 16:37:03.611874 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.611833    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-p8kcp" podStartSLOduration=1.198126082 podStartE2EDuration="3.611818142s" totalImagesPullingTime="2.41369206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:03.61105442 +0000 UTC m=+52.868515845" watchObservedRunningTime="2026-07-30 16:37:03.611818142 +0000 UTC m=+52.869279562"
Jul 30 16:37:03.887585 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.887554    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:37:03.888026 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.887622    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:37:03.888105 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:03.888040    2561 scope.go:117] "RemoveContainer" containerID="87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1"
Jul 30 16:37:03.888238 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:03.888215    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-kskrz_openshift-console-operator(9f5c0566-e219-43bd-8ecf-de5a4e7eed78)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" podUID="9f5c0566-e219-43bd-8ecf-de5a4e7eed78"
Jul 30 16:37:04.245758 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:04.245680    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c_22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09/kube-storage-version-migrator-operator/0.log"
Jul 30 16:37:06.645957 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:06.645924    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:06.646347 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:06.646025    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:06.646347 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:06.646073    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls podName:862368d2-77e5-425b-a497-1caf9e2cb930 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:14.646061373 +0000 UTC m=+63.903522772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djcbt" (UID: "862368d2-77e5-425b-a497-1caf9e2cb930") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:09.470887 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:09.470851    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:09.471339 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:09.470938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:09.471339 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:09.471016    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:25.470995785 +0000 UTC m=+74.728457191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : configmap references non-existent config key: service-ca.crt
Jul 30 16:37:09.471339 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:09.471101    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 16:37:09.471339 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:37:09.471182    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs podName:c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:25.471164287 +0000 UTC m=+74.728625692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs") pod "router-default-6748bdbfdc-jrch2" (UID: "c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6") : secret "router-metrics-certs-default" not found
Jul 30 16:37:10.519354 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:10.519328    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrnv9"
Jul 30 16:37:14.712236 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:14.712188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:14.714564 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:14.714545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/862368d2-77e5-425b-a497-1caf9e2cb930-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djcbt\" (UID: \"862368d2-77e5-425b-a497-1caf9e2cb930\") " pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:14.791088 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:14.791062    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vspgg\""
Jul 30 16:37:14.799329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:14.799311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-djcbt"
Jul 30 16:37:14.919331 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:14.919308    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-djcbt"]
Jul 30 16:37:14.921691 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:14.921653    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod862368d2_77e5_425b_a497_1caf9e2cb930.slice/crio-28c98b2bcac87f6796be9d5da2e9a17af6a3ea744dd301b929793b0a1316a317 WatchSource:0}: Error finding container 28c98b2bcac87f6796be9d5da2e9a17af6a3ea744dd301b929793b0a1316a317: Status 404 returned error can't find the container with id 28c98b2bcac87f6796be9d5da2e9a17af6a3ea744dd301b929793b0a1316a317
Jul 30 16:37:15.633803 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:15.633716    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djcbt" event={"ID":"862368d2-77e5-425b-a497-1caf9e2cb930","Type":"ContainerStarted","Data":"28c98b2bcac87f6796be9d5da2e9a17af6a3ea744dd301b929793b0a1316a317"}
Jul 30 16:37:16.334959 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.334933    2561 scope.go:117] "RemoveContainer" containerID="87705f3a1edeb4917e04a7f725aad4affa2ae0e502a7daeedcf97baca48e6ca1"
Jul 30 16:37:16.640186 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.640108    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:37:16.640337 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.640187    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" event={"ID":"9f5c0566-e219-43bd-8ecf-de5a4e7eed78","Type":"ContainerStarted","Data":"a32a45aa9d41fd683fcd0c4bedd5fd068fcbafe98638097369f5401592c94d7b"}
Jul 30 16:37:16.640553 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.640509    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:37:16.661856 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.661806    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz" podStartSLOduration=20.484893665 podStartE2EDuration="23.661791107s" totalImagesPullingTime="3.176897442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:16.660781163 +0000 UTC m=+65.918242608" watchObservedRunningTime="2026-07-30 16:37:16.661791107 +0000 UTC m=+65.919252527"
Jul 30 16:37:16.932927 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.932817    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:37:16.933096 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.932981    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:37:16.933096 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.933033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:37:16.935559 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.935531    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 16:37:16.935713 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.935531    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 16:37:16.935713 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.935531    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 16:37:16.946151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.946127    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 16:37:16.946285 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.946231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a8a9b3-f13d-4b3a-9729-9bb7b1252705-metrics-certs\") pod \"network-metrics-daemon-8lzgx\" (UID: \"54a8a9b3-f13d-4b3a-9729-9bb7b1252705\") " pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:37:16.946356 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.946303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741ecd44-3b6f-4429-9900-e03e8d8710b9-original-pull-secret\") pod \"global-pull-secret-syncer-mkxgs\" (UID: \"741ecd44-3b6f-4429-9900-e03e8d8710b9\") " pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:37:16.957034 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:16.957010    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcbln\" (UniqueName: \"kubernetes.io/projected/3dbcfb66-0274-4978-b49e-76b2eeb342c0-kube-api-access-mcbln\") pod \"network-check-target-87vzb\" (UID: \"3dbcfb66-0274-4978-b49e-76b2eeb342c0\") " pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:37:17.047605 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.047577    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2m4bh\""
Jul 30 16:37:17.053186 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.053161    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2ksq\""
Jul 30 16:37:17.055176 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.055154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:37:17.055305 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.055252    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkxgs"
Jul 30 16:37:17.061151 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.061132    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8lzgx"
Jul 30 16:37:17.234186 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.234147    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8lzgx"]
Jul 30 16:37:17.240344 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:17.240316    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a8a9b3_f13d_4b3a_9729_9bb7b1252705.slice/crio-61f1e6d4b80370816cd56ddba5bea8a7d5742bf336211f1cc52788d2ea8f9a33 WatchSource:0}: Error finding container 61f1e6d4b80370816cd56ddba5bea8a7d5742bf336211f1cc52788d2ea8f9a33: Status 404 returned error can't find the container with id 61f1e6d4b80370816cd56ddba5bea8a7d5742bf336211f1cc52788d2ea8f9a33
Jul 30 16:37:17.252327 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.252189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mkxgs"]
Jul 30 16:37:17.254920 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:17.254902    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741ecd44_3b6f_4429_9900_e03e8d8710b9.slice/crio-602280d1ed57d46e9a9a189dd58de8b10387aead14e29a92b19f2fed38b9bc6f WatchSource:0}: Error finding container 602280d1ed57d46e9a9a189dd58de8b10387aead14e29a92b19f2fed38b9bc6f: Status 404 returned error can't find the container with id 602280d1ed57d46e9a9a189dd58de8b10387aead14e29a92b19f2fed38b9bc6f
Jul 30 16:37:17.268890 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.268869    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-87vzb"]
Jul 30 16:37:17.273814 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:17.273786    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbcfb66_0274_4978_b49e_76b2eeb342c0.slice/crio-120fa058deea30a1d9c64533c654c449c723cb5f3b0e3f0909092755a110a7b6 WatchSource:0}: Error finding container 120fa058deea30a1d9c64533c654c449c723cb5f3b0e3f0909092755a110a7b6: Status 404 returned error can't find the container with id 120fa058deea30a1d9c64533c654c449c723cb5f3b0e3f0909092755a110a7b6
Jul 30 16:37:17.459386 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.459308    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-kskrz"
Jul 30 16:37:17.647044 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.646999    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-87vzb" event={"ID":"3dbcfb66-0274-4978-b49e-76b2eeb342c0","Type":"ContainerStarted","Data":"120fa058deea30a1d9c64533c654c449c723cb5f3b0e3f0909092755a110a7b6"}
Jul 30 16:37:17.648241 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.648212    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8lzgx" event={"ID":"54a8a9b3-f13d-4b3a-9729-9bb7b1252705","Type":"ContainerStarted","Data":"61f1e6d4b80370816cd56ddba5bea8a7d5742bf336211f1cc52788d2ea8f9a33"}
Jul 30 16:37:17.649394 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:17.649367    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mkxgs" event={"ID":"741ecd44-3b6f-4429-9900-e03e8d8710b9","Type":"ContainerStarted","Data":"602280d1ed57d46e9a9a189dd58de8b10387aead14e29a92b19f2fed38b9bc6f"}
Jul 30 16:37:18.549522 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:18.549467    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:18.552034 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:18.551969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc28418d-5fec-4aac-a992-e57d9832495b-metrics-tls\") pod \"dns-default-99jng\" (UID: \"bc28418d-5fec-4aac-a992-e57d9832495b\") " pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:18.742937 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:18.742908    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59qbk\""
Jul 30 16:37:18.750362 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:18.750335    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:20.655608 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:20.655520    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99jng"]
Jul 30 16:37:20.812714 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:20.812639    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc28418d_5fec_4aac_a992_e57d9832495b.slice/crio-de69612808909f3059aef682f53041ebcd567bacef09ee58de60f372b4e99fc1 WatchSource:0}: Error finding container de69612808909f3059aef682f53041ebcd567bacef09ee58de60f372b4e99fc1: Status 404 returned error can't find the container with id de69612808909f3059aef682f53041ebcd567bacef09ee58de60f372b4e99fc1
Jul 30 16:37:21.670610 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:21.670515    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99jng" event={"ID":"bc28418d-5fec-4aac-a992-e57d9832495b","Type":"ContainerStarted","Data":"de69612808909f3059aef682f53041ebcd567bacef09ee58de60f372b4e99fc1"}
Jul 30 16:37:21.673368 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:21.673334    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djcbt" event={"ID":"862368d2-77e5-425b-a497-1caf9e2cb930","Type":"ContainerStarted","Data":"e2c619ffc0c4fc9ddef2abd7eda234f6bee447795fa6b4a18ad253d323572820"}
Jul 30 16:37:21.679037 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:21.678309    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8lzgx" event={"ID":"54a8a9b3-f13d-4b3a-9729-9bb7b1252705","Type":"ContainerStarted","Data":"d0c3e8429e179988636ea03eefd7cd000a76693e4f43419b5b600296ece62eb9"}
Jul 30 16:37:21.679037 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:21.678363    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8lzgx" event={"ID":"54a8a9b3-f13d-4b3a-9729-9bb7b1252705","Type":"ContainerStarted","Data":"ac2edaf4eb0caf9431827fff8dd53c0f9b2e266b0a44b804940015e5cf36d43f"}
Jul 30 16:37:22.553708 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.553557    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8lzgx" podStartSLOduration=67.956882737 podStartE2EDuration="1m11.553531356s" totalImagesPullingTime="3.596648619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:21.700194188 +0000 UTC m=+70.957655610" watchObservedRunningTime="2026-07-30 16:37:22.553531356 +0000 UTC m=+71.810992780"
Jul 30 16:37:22.554622 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.554595    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"]
Jul 30 16:37:22.557069 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.557048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.560627 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.560449    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 16:37:22.560627 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.560586    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 16:37:22.561449 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.561324    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 16:37:22.561449 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.561334    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 16:37:22.561449 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.561348    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 16:37:22.561449 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.561367    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 16:37:22.561735 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.561701    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 16:37:22.575834 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.575779    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"]
Jul 30 16:37:22.684941 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.684914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77t2\" (UniqueName: \"kubernetes.io/projected/878b5283-096d-4950-8f7e-55ab7583fc1b-kube-api-access-n77t2\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.685389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.684950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.685389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.685000    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.685389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.685039    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.685389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.685095    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.685389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.685142    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5283-096d-4950-8f7e-55ab7583fc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.785752 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785718    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5283-096d-4950-8f7e-55ab7583fc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.785931 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785758    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77t2\" (UniqueName: \"kubernetes.io/projected/878b5283-096d-4950-8f7e-55ab7583fc1b-kube-api-access-n77t2\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.785931 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785790    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.785931 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785832    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.785931 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.786136 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.785955    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.786588 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.786561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5283-096d-4950-8f7e-55ab7583fc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.789083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.789028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-ca\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.789083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.789056    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.789083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.789060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.789083 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.789063    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/878b5283-096d-4950-8f7e-55ab7583fc1b-hub\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.800960 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.800940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77t2\" (UniqueName: \"kubernetes.io/projected/878b5283-096d-4950-8f7e-55ab7583fc1b-kube-api-access-n77t2\") pod \"cluster-proxy-proxy-agent-6b9dc4f689-nb862\" (UID: \"878b5283-096d-4950-8f7e-55ab7583fc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:22.876686 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:22.876650    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"
Jul 30 16:37:25.510286 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.510255    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:25.510782 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.510306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:25.511049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.511018    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-service-ca-bundle\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:25.512574 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.512553    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6-metrics-certs\") pod \"router-default-6748bdbfdc-jrch2\" (UID: \"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6\") " pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:25.671936 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.671907    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-47bt9\""
Jul 30 16:37:25.679829 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:25.679806    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:26.730958 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:26.730935    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862"]
Jul 30 16:37:26.849518 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:26.848687    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6748bdbfdc-jrch2"]
Jul 30 16:37:26.852792 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:26.852713    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ec96c4_2282_4a6c_acfa_2a2111cf2ce6.slice/crio-e5e9016c8a0e1e8c2b7d6686d78a5bfdc7066d2873326ca523c1fad7e04a5a40 WatchSource:0}: Error finding container e5e9016c8a0e1e8c2b7d6686d78a5bfdc7066d2873326ca523c1fad7e04a5a40: Status 404 returned error can't find the container with id e5e9016c8a0e1e8c2b7d6686d78a5bfdc7066d2873326ca523c1fad7e04a5a40
Jul 30 16:37:27.705898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.705052    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-87vzb" event={"ID":"3dbcfb66-0274-4978-b49e-76b2eeb342c0","Type":"ContainerStarted","Data":"dda0bafe8e4efaab81ef655077f15e325bc5b50863570998367916d39a0c1a55"}
Jul 30 16:37:27.705898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.705842    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:37:27.708652 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.708591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djcbt" event={"ID":"862368d2-77e5-425b-a497-1caf9e2cb930","Type":"ContainerStarted","Data":"602fd88f9a4b3a04e8910787e345ebcf6309faa39d1e388af28ceac4a273aa7e"}
Jul 30 16:37:27.711721 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.711263    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mkxgs" event={"ID":"741ecd44-3b6f-4429-9900-e03e8d8710b9","Type":"ContainerStarted","Data":"7bd300ed2d8cd2acfa6e8ad241888af0d21bb420f0b9039ea37890b53859cea4"}
Jul 30 16:37:27.715109 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.715058    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99jng" event={"ID":"bc28418d-5fec-4aac-a992-e57d9832495b","Type":"ContainerStarted","Data":"4a962c2a85ed7b025d9655926ac50189b42a0c8df7800680dd7725baac123521"}
Jul 30 16:37:27.715109 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.715107    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99jng" event={"ID":"bc28418d-5fec-4aac-a992-e57d9832495b","Type":"ContainerStarted","Data":"14530324fedc3033ddb425cae96944bc315a05f6e86abdf370e1d98437bd110a"}
Jul 30 16:37:27.715660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.715639    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:27.717528 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.717504    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6748bdbfdc-jrch2" event={"ID":"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6","Type":"ContainerStarted","Data":"9517051fa006e7435510e62c67cb5770529b8fe757a253b741106afde2895ec9"}
Jul 30 16:37:27.717627 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.717547    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6748bdbfdc-jrch2" event={"ID":"c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6","Type":"ContainerStarted","Data":"e5e9016c8a0e1e8c2b7d6686d78a5bfdc7066d2873326ca523c1fad7e04a5a40"}
Jul 30 16:37:27.719521 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.719497    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862" event={"ID":"878b5283-096d-4950-8f7e-55ab7583fc1b","Type":"ContainerStarted","Data":"efcb3c557f020679a69c2337299bea14e49782c8721d0c498274326b7cfbfde1"}
Jul 30 16:37:27.724844 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.724695    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-87vzb" podStartSLOduration=67.958281514 podStartE2EDuration="1m16.724676538s" totalImagesPullingTime="8.766395024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:27.723661088 +0000 UTC m=+76.981122536" watchObservedRunningTime="2026-07-30 16:37:27.724676538 +0000 UTC m=+76.982137961"
Jul 30 16:37:27.745241 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.745044    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6748bdbfdc-jrch2" podStartSLOduration=34.745027274 podStartE2EDuration="34.745027274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:27.744693462 +0000 UTC m=+77.002154911" watchObservedRunningTime="2026-07-30 16:37:27.745027274 +0000 UTC m=+77.002488695"
Jul 30 16:37:27.763673 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.761738    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mkxgs" podStartSLOduration=67.322313033 podStartE2EDuration="1m16.76171844s" totalImagesPullingTime="9.439405407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:27.761275085 +0000 UTC m=+77.018736530" watchObservedRunningTime="2026-07-30 16:37:27.76171844 +0000 UTC m=+77.019179861"
Jul 30 16:37:27.780877 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:27.780829    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-99jng" podStartSLOduration=35.899054423 podStartE2EDuration="41.780810147s" totalImagesPullingTime="5.881755724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:27.779526725 +0000 UTC m=+77.036988166" watchObservedRunningTime="2026-07-30 16:37:27.780810147 +0000 UTC m=+77.038271568"
Jul 30 16:37:28.680839 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:28.680760    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:28.683867 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:28.683755    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:28.725805 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:28.725772    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:28.727270 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:28.727139    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6748bdbfdc-jrch2"
Jul 30 16:37:32.740857 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:32.740723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djcbt" event={"ID":"862368d2-77e5-425b-a497-1caf9e2cb930","Type":"ContainerStarted","Data":"1ec5f491053866a45ea5f222bafc0a4202528042772d400bfb587c725a082b67"}
Jul 30 16:37:32.742686 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:32.742663    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862" event={"ID":"878b5283-096d-4950-8f7e-55ab7583fc1b","Type":"ContainerStarted","Data":"4add10f842986ed7694606120f98a1a356486bc7d541c2612186e2e15a4f8715"}
Jul 30 16:37:32.742803 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:32.742692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862" event={"ID":"878b5283-096d-4950-8f7e-55ab7583fc1b","Type":"ContainerStarted","Data":"232eadf2bc67313fa36de8e2684e47c993cfdaaa2ae31580c583c4e517b2c3c7"}
Jul 30 16:37:32.742803 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:32.742704    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862" event={"ID":"878b5283-096d-4950-8f7e-55ab7583fc1b","Type":"ContainerStarted","Data":"978e168fcb9a84c6890bad077ba6be3ec8f08a3b78e201132ac783131a5c4966"}
Jul 30 16:37:32.774518 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:32.774459    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-djcbt" podStartSLOduration=17.88221887 podStartE2EDuration="34.774446711s" totalImagesPullingTime="16.892227841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:32.7731937 +0000 UTC m=+82.030655122" watchObservedRunningTime="2026-07-30 16:37:32.774446711 +0000 UTC m=+82.031908130"
Jul 30 16:37:38.730139 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:38.730111    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-99jng"
Jul 30 16:37:38.746348 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:38.746300    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b9dc4f689-nb862" podStartSLOduration=11.518698863000001 podStartE2EDuration="16.746287426s" totalImagesPullingTime="5.227588563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:32.804704153 +0000 UTC m=+82.062165576" watchObservedRunningTime="2026-07-30 16:37:38.746287426 +0000 UTC m=+88.003748845"
Jul 30 16:37:51.489339 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.489304    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-plgb2"]
Jul 30 16:37:51.495691 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.495671    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.499820 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.499798    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 16:37:51.499960 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.499852    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 16:37:51.499960 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.499889    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 16:37:51.499960 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.499908    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzhcm\""
Jul 30 16:37:51.499960 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.499853    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 16:37:51.500223 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.500060    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 16:37:51.500223 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.500157    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 16:37:51.587947 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.587918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkg2x\" (UniqueName: \"kubernetes.io/projected/7f66f702-5be0-43fd-b57c-1a1dd8219483-kube-api-access-xkg2x\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.587962    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-root\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.587984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-accelerators-collector-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588035    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-wtmp\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588275 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-sys\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588275 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588166    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-textfile\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588275 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-metrics-client-ca\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.588275 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.588248    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-tls\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689433 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-wtmp\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689433 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-sys\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689503    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-textfile\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689530    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-metrics-client-ca\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689552    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-sys\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689604    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-wtmp\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689660 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-tls\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689951 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkg2x\" (UniqueName: \"kubernetes.io/projected/7f66f702-5be0-43fd-b57c-1a1dd8219483-kube-api-access-xkg2x\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689951 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-root\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689951 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689735    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-accelerators-collector-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.689951 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.689891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f66f702-5be0-43fd-b57c-1a1dd8219483-root\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.690140 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.690016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-textfile\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.690215 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.690195    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-metrics-client-ca\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.690268 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.690216    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-accelerators-collector-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.691886 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.691854    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-tls\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.691984 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.691948    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f66f702-5be0-43fd-b57c-1a1dd8219483-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.698763 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.698739    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkg2x\" (UniqueName: \"kubernetes.io/projected/7f66f702-5be0-43fd-b57c-1a1dd8219483-kube-api-access-xkg2x\") pod \"node-exporter-plgb2\" (UID: \"7f66f702-5be0-43fd-b57c-1a1dd8219483\") " pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.804569 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:51.804499    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-plgb2"
Jul 30 16:37:51.813638 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:37:51.813610    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f66f702_5be0_43fd_b57c_1a1dd8219483.slice/crio-3dc7ac2a9aa07ca2f5b1adf681d47068703aed0ec2a311510c63df11b04806b1 WatchSource:0}: Error finding container 3dc7ac2a9aa07ca2f5b1adf681d47068703aed0ec2a311510c63df11b04806b1: Status 404 returned error can't find the container with id 3dc7ac2a9aa07ca2f5b1adf681d47068703aed0ec2a311510c63df11b04806b1
Jul 30 16:37:52.811049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:52.811009    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-plgb2" event={"ID":"7f66f702-5be0-43fd-b57c-1a1dd8219483","Type":"ContainerStarted","Data":"3dc7ac2a9aa07ca2f5b1adf681d47068703aed0ec2a311510c63df11b04806b1"}
Jul 30 16:37:53.816332 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:53.816295    2561 generic.go:356] "Generic (PLEG): container finished" podID="7f66f702-5be0-43fd-b57c-1a1dd8219483" containerID="74010215037b123041f177d976b35d1cf6d46980e4018f45733a5a23aaa96780" exitCode=0
Jul 30 16:37:53.816740 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:53.816365    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-plgb2" event={"ID":"7f66f702-5be0-43fd-b57c-1a1dd8219483","Type":"ContainerDied","Data":"74010215037b123041f177d976b35d1cf6d46980e4018f45733a5a23aaa96780"}
Jul 30 16:37:54.822689 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:54.822655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-plgb2" event={"ID":"7f66f702-5be0-43fd-b57c-1a1dd8219483","Type":"ContainerStarted","Data":"2157f75be5339ab7e52dd8322bccdbf7a18f76f545d780a724c704b32cabba91"}
Jul 30 16:37:54.823098 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:54.822714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-plgb2" event={"ID":"7f66f702-5be0-43fd-b57c-1a1dd8219483","Type":"ContainerStarted","Data":"4975c0116e5f190abe1736ce64b0ec7951d6dadbb9998a22e17bbfdf3c7b02ba"}
Jul 30 16:37:54.843000 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:54.842930    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-plgb2" podStartSLOduration=2.664396067 podStartE2EDuration="3.842914298s" totalImagesPullingTime="1.116001072s" totalInitContainerRuntime="62.517159ms" isStatefulPod=true podCreationTimestamp="2026-07-30 16:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:54.84147133 +0000 UTC m=+104.098932772" watchObservedRunningTime="2026-07-30 16:37:54.842914298 +0000 UTC m=+104.100375723"
Jul 30 16:37:59.729949 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:37:59.729915    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-87vzb"
Jul 30 16:38:07.866868 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:38:07.866835    2561 generic.go:356] "Generic (PLEG): container finished" podID="22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09" containerID="c0041b7b1d6913c81cfcd7641a2a2a811ae024a0c9fcdce8342dccc277f19dd0" exitCode=0
Jul 30 16:38:07.867250 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:38:07.866906    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c" event={"ID":"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09","Type":"ContainerDied","Data":"c0041b7b1d6913c81cfcd7641a2a2a811ae024a0c9fcdce8342dccc277f19dd0"}
Jul 30 16:38:07.867250 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:38:07.867199    2561 scope.go:117] "RemoveContainer" containerID="c0041b7b1d6913c81cfcd7641a2a2a811ae024a0c9fcdce8342dccc277f19dd0"
Jul 30 16:38:08.871983 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:38:08.871951    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c" event={"ID":"22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09","Type":"ContainerStarted","Data":"055764b50fed709e49454173ce47191312a2dea69ceeda2679c06f662bd03ff3"}
Jul 30 16:41:11.116432 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:41:11.116405    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:41:11.121953 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:41:11.121927    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:41:11.122105 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:41:11.122076    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:41:11.126218 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:41:11.126199    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 16:41:11.128010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:41:11.127990    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:42:18.212338 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.212299    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"]
Jul 30 16:42:18.214257 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.214238    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.218680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.218658    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 16:42:18.219559 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.219539    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sth8j\""
Jul 30 16:42:18.219677 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.219599    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 16:42:18.226371 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.226350    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"]
Jul 30 16:42:18.297946 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.297909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.298167 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.297972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp55w\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-kube-api-access-gp55w\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.399024 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.398993    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp55w\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-kube-api-access-gp55w\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.399189 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.399047    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.408277 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.408240    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.408386 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.408349    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp55w\" (UniqueName: \"kubernetes.io/projected/01aeca00-19f5-4ebd-862f-010b988481bf-kube-api-access-gp55w\") pod \"cert-manager-cainjector-559cdd8c87-d84lc\" (UID: \"01aeca00-19f5-4ebd-862f-010b988481bf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.523399 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.523315    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"
Jul 30 16:42:18.652120 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.652090    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d84lc"]
Jul 30 16:42:18.655718 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:42:18.655688    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01aeca00_19f5_4ebd_862f_010b988481bf.slice/crio-0b202e35e64af09dcbb0a247e17bf1b8d22f4f371885099551e6b25eecf074a5 WatchSource:0}: Error finding container 0b202e35e64af09dcbb0a247e17bf1b8d22f4f371885099551e6b25eecf074a5: Status 404 returned error can't find the container with id 0b202e35e64af09dcbb0a247e17bf1b8d22f4f371885099551e6b25eecf074a5
Jul 30 16:42:18.657525 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.657510    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:42:18.685697 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:18.685667    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc" event={"ID":"01aeca00-19f5-4ebd-862f-010b988481bf","Type":"ContainerStarted","Data":"0b202e35e64af09dcbb0a247e17bf1b8d22f4f371885099551e6b25eecf074a5"}
Jul 30 16:42:22.704947 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:22.704856    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc" event={"ID":"01aeca00-19f5-4ebd-862f-010b988481bf","Type":"ContainerStarted","Data":"a0d9a384f671587148892444b414b9ae5d18bd79b902485adef29d55393df119"}
Jul 30 16:42:22.725807 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:22.725762    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d84lc" podStartSLOduration=0.938501418 podStartE2EDuration="4.725745017s" totalImagesPullingTime="3.787243599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:42:22.724249483 +0000 UTC m=+371.981710902" watchObservedRunningTime="2026-07-30 16:42:22.725745017 +0000 UTC m=+371.983206437"
Jul 30 16:42:48.478424 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.478392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"]
Jul 30 16:42:48.481201 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.481185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.495980 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.495960    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 16:42:48.496096 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.496053    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 16:42:48.496161 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.496139    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 16:42:48.496270 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.496243    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lp2wm\""
Jul 30 16:42:48.496753 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.496738    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 16:42:48.502598 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.502570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"]
Jul 30 16:42:48.509788 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.509755    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.509898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.509797    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.509898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.509862    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.510003 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.509939    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.510065 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.510003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5k2t\" (UniqueName: \"kubernetes.io/projected/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-kube-api-access-p5k2t\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610445 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610420    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610573 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610458    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610573 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5k2t\" (UniqueName: \"kubernetes.io/projected/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-kube-api-access-p5k2t\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610573 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610726 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610588    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.610961 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.610940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.612771 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.612751    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.613055 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.613035    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.613055 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.613047    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.659446 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.659418    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5k2t\" (UniqueName: \"kubernetes.io/projected/1f8fd8bf-afc1-4c0f-ba0a-caf458418db8-kube-api-access-p5k2t\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-49vbm\" (UID: \"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.790679 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.790606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:48.923328 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:48.923204    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"]
Jul 30 16:42:48.926007 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:42:48.925987    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8fd8bf_afc1_4c0f_ba0a_caf458418db8.slice/crio-dee11d8ffaad02f1ddd25f1165de4766763baa427f1d6f06502ba89d2c192ad8 WatchSource:0}: Error finding container dee11d8ffaad02f1ddd25f1165de4766763baa427f1d6f06502ba89d2c192ad8: Status 404 returned error can't find the container with id dee11d8ffaad02f1ddd25f1165de4766763baa427f1d6f06502ba89d2c192ad8
Jul 30 16:42:49.803993 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:49.803942    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm" event={"ID":"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8","Type":"ContainerStarted","Data":"dee11d8ffaad02f1ddd25f1165de4766763baa427f1d6f06502ba89d2c192ad8"}
Jul 30 16:42:52.817748 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:52.817710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm" event={"ID":"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8","Type":"ContainerStarted","Data":"4db8ef900d8ad6bec12d850b010b1320b17f82c49119f9279967aa3b0bc9317a"}
Jul 30 16:42:53.823014 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:53.822980    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f8fd8bf-afc1-4c0f-ba0a-caf458418db8" containerID="4db8ef900d8ad6bec12d850b010b1320b17f82c49119f9279967aa3b0bc9317a" exitCode=0
Jul 30 16:42:53.823419 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:53.823041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm" event={"ID":"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8","Type":"ContainerDied","Data":"4db8ef900d8ad6bec12d850b010b1320b17f82c49119f9279967aa3b0bc9317a"}
Jul 30 16:42:54.828675 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:54.828638    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm" event={"ID":"1f8fd8bf-afc1-4c0f-ba0a-caf458418db8","Type":"ContainerStarted","Data":"a0e2b788fa7904793c596c2a2e5f553931c5cb406e70d8079679ab09984eb788"}
Jul 30 16:42:54.829093 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:54.828704    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:42:54.853446 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:42:54.853388    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm" podStartSLOduration=1.909977913 podStartE2EDuration="6.853369385s" totalImagesPullingTime="3.910218881s" totalInitContainerRuntime="1.033172591s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:42:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:42:54.851210055 +0000 UTC m=+404.108671494" watchObservedRunningTime="2026-07-30 16:42:54.853369385 +0000 UTC m=+404.110830805"
Jul 30 16:43:05.834455 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:05.834427    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-49vbm"
Jul 30 16:43:13.962601 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.962570    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"]
Jul 30 16:43:13.967135 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.967115    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:13.969379 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.969361    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 16:43:13.969504 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.969383    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qdc4m\""
Jul 30 16:43:13.970074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.970057    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 16:43:13.977751 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:13.977730    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"]
Jul 30 16:43:14.098113 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.098085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.098260 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.098130    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c298h\" (UniqueName: \"kubernetes.io/projected/f01341b6-8c3a-4278-90d0-e59fb8b0da85-kube-api-access-c298h\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.098260 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.098153    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tmp\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.124184 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.124159    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tn6pp"]
Jul 30 16:43:14.126994 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.126980    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.129415 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.129397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 30 16:43:14.129897 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.129885    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9gr2p\""
Jul 30 16:43:14.139910 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.139891    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tn6pp"]
Jul 30 16:43:14.198779 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.198752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.198936 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.198786    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxk9s\" (UniqueName: \"kubernetes.io/projected/de85fd46-7924-43fc-8952-f02d11366e71-kube-api-access-fxk9s\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.198936 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.198813    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.198936 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.198891    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c298h\" (UniqueName: \"kubernetes.io/projected/f01341b6-8c3a-4278-90d0-e59fb8b0da85-kube-api-access-c298h\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.199157 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.199030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tmp\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.201249 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.201226    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tmp\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.201453 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.201426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01341b6-8c3a-4278-90d0-e59fb8b0da85-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.206807 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.206785    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c298h\" (UniqueName: \"kubernetes.io/projected/f01341b6-8c3a-4278-90d0-e59fb8b0da85-kube-api-access-c298h\") pod \"kube-auth-proxy-bbc57f5f6-5jrdw\" (UID: \"f01341b6-8c3a-4278-90d0-e59fb8b0da85\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.278753 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.278687    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"
Jul 30 16:43:14.300001 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.299970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxk9s\" (UniqueName: \"kubernetes.io/projected/de85fd46-7924-43fc-8952-f02d11366e71-kube-api-access-fxk9s\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.300135 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.300016    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.300135 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:43:14.300114    2561 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 30 16:43:14.300236 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:43:14.300182    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert podName:de85fd46-7924-43fc-8952-f02d11366e71 nodeName:}" failed. No retries permitted until 2026-07-30 16:43:14.800166754 +0000 UTC m=+424.057628156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert") pod "odh-model-controller-7998c67dc4-tn6pp" (UID: "de85fd46-7924-43fc-8952-f02d11366e71") : secret "odh-model-controller-webhook-cert" not found
Jul 30 16:43:14.308796 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.308770    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxk9s\" (UniqueName: \"kubernetes.io/projected/de85fd46-7924-43fc-8952-f02d11366e71-kube-api-access-fxk9s\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.412138 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.412113    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw"]
Jul 30 16:43:14.413381 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:14.413358    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01341b6_8c3a_4278_90d0_e59fb8b0da85.slice/crio-ee2cf5c4226364f03978e007c2647fe22e050570e7252b8de6eb9ffafdf40a96 WatchSource:0}: Error finding container ee2cf5c4226364f03978e007c2647fe22e050570e7252b8de6eb9ffafdf40a96: Status 404 returned error can't find the container with id ee2cf5c4226364f03978e007c2647fe22e050570e7252b8de6eb9ffafdf40a96
Jul 30 16:43:14.804706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.804678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.806877 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.806856    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de85fd46-7924-43fc-8952-f02d11366e71-cert\") pod \"odh-model-controller-7998c67dc4-tn6pp\" (UID: \"de85fd46-7924-43fc-8952-f02d11366e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:14.901689 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.901646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw" event={"ID":"f01341b6-8c3a-4278-90d0-e59fb8b0da85","Type":"ContainerStarted","Data":"ee2cf5c4226364f03978e007c2647fe22e050570e7252b8de6eb9ffafdf40a96"}
Jul 30 16:43:14.943569 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.943540    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-q9m9w"]
Jul 30 16:43:14.947782 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.947766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:14.950561 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.950542    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-zjbbh\""
Jul 30 16:43:14.950756 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.950745    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 30 16:43:14.972454 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:14.972425    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-q9m9w"]
Jul 30 16:43:15.006454 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.006426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.006598 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.006495    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmzh\" (UniqueName: \"kubernetes.io/projected/637a960b-22d5-47bf-af57-92486c952828-kube-api-access-jdmzh\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.036435 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.036412    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:15.106978 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.106943    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmzh\" (UniqueName: \"kubernetes.io/projected/637a960b-22d5-47bf-af57-92486c952828-kube-api-access-jdmzh\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.107084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.107055    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.107237 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:43:15.107217    2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 30 16:43:15.107324 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:43:15.107317    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs podName:637a960b-22d5-47bf-af57-92486c952828 nodeName:}" failed. No retries permitted until 2026-07-30 16:43:15.607286786 +0000 UTC m=+424.864748198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs") pod "model-serving-api-54d89bd5fc-q9m9w" (UID: "637a960b-22d5-47bf-af57-92486c952828") : secret "model-serving-api-tls" not found
Jul 30 16:43:15.120750 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.120731    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmzh\" (UniqueName: \"kubernetes.io/projected/637a960b-22d5-47bf-af57-92486c952828-kube-api-access-jdmzh\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.184663 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.184624    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tn6pp"]
Jul 30 16:43:15.188050 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:15.188020    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde85fd46_7924_43fc_8952_f02d11366e71.slice/crio-aa374859d0c06d726d94beba1a95366442322c05496770c6a5f52595e0b0ccc9 WatchSource:0}: Error finding container aa374859d0c06d726d94beba1a95366442322c05496770c6a5f52595e0b0ccc9: Status 404 returned error can't find the container with id aa374859d0c06d726d94beba1a95366442322c05496770c6a5f52595e0b0ccc9
Jul 30 16:43:15.407859 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.407782    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:15.411218 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.411194    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.414096 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.413963    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 16:43:15.414096 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.413980    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 16:43:15.414290 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.414256    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nq667\""
Jul 30 16:43:15.414290 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.414263    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 16:43:15.414389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.414314    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 16:43:15.429184 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.429165    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:15.510992 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.510962    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511160 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511000    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511160 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511025    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511160 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511077    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511160 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511148    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511374 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511178    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.511374 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.511202    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9rqd\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612359 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612331    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.612359 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612365    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612614 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612383    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612614 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612535    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9rqd\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612720 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612720 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612733    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.612827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.612760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.613037 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.613017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.615032 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.615001    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.615329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.615303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.615426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.615359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637a960b-22d5-47bf-af57-92486c952828-tls-certs\") pod \"model-serving-api-54d89bd5fc-q9m9w\" (UID: \"637a960b-22d5-47bf-af57-92486c952828\") " pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.615426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.615368    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.615426 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.615416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.621765 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.621743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.623665 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.623644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9rqd\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd\") pod \"istiod-openshift-gateway-7b7995d5cd-tpkwk\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.721634 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.721560    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:15.860569 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.860536    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:15.876723 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.876689    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:15.878828 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:15.878802    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1ec96_cba5_4c36_a300_cb6ede179087.slice/crio-dadef078e4dca10a96ed638c6b05b8f6d50ea1add5a87272f780d0bd3ae58520 WatchSource:0}: Error finding container dadef078e4dca10a96ed638c6b05b8f6d50ea1add5a87272f780d0bd3ae58520: Status 404 returned error can't find the container with id dadef078e4dca10a96ed638c6b05b8f6d50ea1add5a87272f780d0bd3ae58520
Jul 30 16:43:15.907787 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.907751    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp" event={"ID":"de85fd46-7924-43fc-8952-f02d11366e71","Type":"ContainerStarted","Data":"aa374859d0c06d726d94beba1a95366442322c05496770c6a5f52595e0b0ccc9"}
Jul 30 16:43:15.909548 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:15.909520    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" event={"ID":"33c1ec96-cba5-4c36-a300-cb6ede179087","Type":"ContainerStarted","Data":"dadef078e4dca10a96ed638c6b05b8f6d50ea1add5a87272f780d0bd3ae58520"}
Jul 30 16:43:16.017768 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:16.017654    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-q9m9w"]
Jul 30 16:43:16.020950 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:16.020915    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637a960b_22d5_47bf_af57_92486c952828.slice/crio-6f6200f6351c71290550a811d76a462312e3f464a7628160be349345bfb55884 WatchSource:0}: Error finding container 6f6200f6351c71290550a811d76a462312e3f464a7628160be349345bfb55884: Status 404 returned error can't find the container with id 6f6200f6351c71290550a811d76a462312e3f464a7628160be349345bfb55884
Jul 30 16:43:16.919243 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:16.919167    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w" event={"ID":"637a960b-22d5-47bf-af57-92486c952828","Type":"ContainerStarted","Data":"6f6200f6351c71290550a811d76a462312e3f464a7628160be349345bfb55884"}
Jul 30 16:43:19.939536 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.939503    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw" event={"ID":"f01341b6-8c3a-4278-90d0-e59fb8b0da85","Type":"ContainerStarted","Data":"4f783fcbb41ab0b7d6902a054415904b34f26d0b2444d927da59d4ce3cf30533"}
Jul 30 16:43:19.941171 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.941138    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w" event={"ID":"637a960b-22d5-47bf-af57-92486c952828","Type":"ContainerStarted","Data":"352617ee5b3e49b1c78a67cbd1aa3b6d647795f2ef4f78b5ff73f02161bbd4bb"}
Jul 30 16:43:19.941321 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.941221    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:43:19.942758 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.942736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp" event={"ID":"de85fd46-7924-43fc-8952-f02d11366e71","Type":"ContainerStarted","Data":"0c42355a916e05fe3958f27976be98ace81d787c9eb380ebd561c65dcdb50389"}
Jul 30 16:43:19.942881 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.942847    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:19.957302 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.957229    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-5jrdw" podStartSLOduration=2.220054253 podStartE2EDuration="6.95721286s" totalImagesPullingTime="4.737158607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:19.956421091 +0000 UTC m=+429.213882505" watchObservedRunningTime="2026-07-30 16:43:19.95721286 +0000 UTC m=+429.214674280"
Jul 30 16:43:19.976391 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.976345    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w" podStartSLOduration=2.846643019 podStartE2EDuration="5.976327419s" totalImagesPullingTime="3.1296844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:19.975013693 +0000 UTC m=+429.232475130" watchObservedRunningTime="2026-07-30 16:43:19.976327419 +0000 UTC m=+429.233788838"
Jul 30 16:43:19.994247 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:19.994196    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp" podStartSLOduration=2.020992734 podStartE2EDuration="5.994182576s" totalImagesPullingTime="3.973189842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:19.992240779 +0000 UTC m=+429.249702224" watchObservedRunningTime="2026-07-30 16:43:19.994182576 +0000 UTC m=+429.251644009"
Jul 30 16:43:20.472621 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:20.472578    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 16:43:20.472737 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:20.472656    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 16:43:20.948477 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:20.948442    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" event={"ID":"33c1ec96-cba5-4c36-a300-cb6ede179087","Type":"ContainerStarted","Data":"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"}
Jul 30 16:43:21.953329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:21.953296    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:21.954898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:21.954868    2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-tpkwk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 16:43:21.955009 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:21.954927    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 16:43:22.957839 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:22.957808    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:23.056901 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:23.056852    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" podStartSLOduration=3.465837214 podStartE2EDuration="8.056836316s" totalImagesPullingTime="4.590999102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:21.016362255 +0000 UTC m=+430.273823674" watchObservedRunningTime="2026-07-30 16:43:23.056836316 +0000 UTC m=+432.314297736"
Jul 30 16:43:26.198575 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.198536    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"]
Jul 30 16:43:26.205345 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.205321    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.207338 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.207314    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"]
Jul 30 16:43:26.209687 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.209672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.210834 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.210815    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 16:43:26.210916 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.210819    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-72nwf\""
Jul 30 16:43:26.213387 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.213363    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pnpn4\""
Jul 30 16:43:26.213517 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.213476    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 30 16:43:26.228087 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.228061    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"]
Jul 30 16:43:26.249606 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.249578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"]
Jul 30 16:43:26.303233 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.303206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbx2\" (UniqueName: \"kubernetes.io/projected/361f5299-762b-4cf5-8144-dd09e4c68bbd-kube-api-access-brbx2\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.303379 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.303255    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfbt2\" (UniqueName: \"kubernetes.io/projected/41a88185-9b73-4109-ab1c-34da262e55de-kube-api-access-gfbt2\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.303379 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.303299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/361f5299-762b-4cf5-8144-dd09e4c68bbd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.303379 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.303316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41a88185-9b73-4109-ab1c-34da262e55de-cert\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.404576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.404549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/361f5299-762b-4cf5-8144-dd09e4c68bbd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.404576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.404578    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41a88185-9b73-4109-ab1c-34da262e55de-cert\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.404763 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.404608    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbx2\" (UniqueName: \"kubernetes.io/projected/361f5299-762b-4cf5-8144-dd09e4c68bbd-kube-api-access-brbx2\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.404763 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.404645    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfbt2\" (UniqueName: \"kubernetes.io/projected/41a88185-9b73-4109-ab1c-34da262e55de-kube-api-access-gfbt2\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.407111 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.407078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41a88185-9b73-4109-ab1c-34da262e55de-cert\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.407111 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.407078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/361f5299-762b-4cf5-8144-dd09e4c68bbd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.432567 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.432547    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfbt2\" (UniqueName: \"kubernetes.io/projected/41a88185-9b73-4109-ab1c-34da262e55de-kube-api-access-gfbt2\") pod \"kserve-controller-manager-d6f4b65b8-bf2lt\" (UID: \"41a88185-9b73-4109-ab1c-34da262e55de\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.448519 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.448498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbx2\" (UniqueName: \"kubernetes.io/projected/361f5299-762b-4cf5-8144-dd09e4c68bbd-kube-api-access-brbx2\") pod \"llmisvc-controller-manager-777c6bd7d5-mcgv4\" (UID: \"361f5299-762b-4cf5-8144-dd09e4c68bbd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.517677 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.517612    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:26.524747 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.524719    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:26.666674 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.666537    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"]
Jul 30 16:43:26.668783 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:26.668755    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a88185_9b73_4109_ab1c_34da262e55de.slice/crio-e9963449f3d44f9f88841aa4254b5fb0eca9a7115833087f28bad0e9090bb6e2 WatchSource:0}: Error finding container e9963449f3d44f9f88841aa4254b5fb0eca9a7115833087f28bad0e9090bb6e2: Status 404 returned error can't find the container with id e9963449f3d44f9f88841aa4254b5fb0eca9a7115833087f28bad0e9090bb6e2
Jul 30 16:43:26.693894 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.693877    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"]
Jul 30 16:43:26.695573 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:43:26.695547    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod361f5299_762b_4cf5_8144_dd09e4c68bbd.slice/crio-6a64a72ab78723ec6c81d6a46c92288ff0ed4944bafee2f8a9c02c552e600b8a WatchSource:0}: Error finding container 6a64a72ab78723ec6c81d6a46c92288ff0ed4944bafee2f8a9c02c552e600b8a: Status 404 returned error can't find the container with id 6a64a72ab78723ec6c81d6a46c92288ff0ed4944bafee2f8a9c02c552e600b8a
Jul 30 16:43:26.974993 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.974962    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt" event={"ID":"41a88185-9b73-4109-ab1c-34da262e55de","Type":"ContainerStarted","Data":"e9963449f3d44f9f88841aa4254b5fb0eca9a7115833087f28bad0e9090bb6e2"}
Jul 30 16:43:26.976062 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:26.976036    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4" event={"ID":"361f5299-762b-4cf5-8144-dd09e4c68bbd","Type":"ContainerStarted","Data":"6a64a72ab78723ec6c81d6a46c92288ff0ed4944bafee2f8a9c02c552e600b8a"}
Jul 30 16:43:28.210607 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.210577    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:28.211066 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.210774    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerName="discovery" containerID="cri-o://34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336" gracePeriod=30
Jul 30 16:43:28.464157 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.464103    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:28.519759 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519731    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.519920 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519774    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.519920 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519809    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.519920 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519845    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.519920 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519870    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9rqd\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.520154 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519949    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.520154 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.519973    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token\") pod \"33c1ec96-cba5-4c36-a300-cb6ede179087\" (UID: \"33c1ec96-cba5-4c36-a300-cb6ede179087\") "
Jul 30 16:43:28.521053 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.520684    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:43:28.522448 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.522421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts" (OuterVolumeSpecName: "cacerts") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:43:28.522925 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.522886    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token" (OuterVolumeSpecName: "istio-token") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:43:28.523052 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.523028    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:43:28.523695 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.523662    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs" (OuterVolumeSpecName: "local-certs") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:43:28.524439 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.524412    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:43:28.524743 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.524717    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd" (OuterVolumeSpecName: "kube-api-access-s9rqd") pod "33c1ec96-cba5-4c36-a300-cb6ede179087" (UID: "33c1ec96-cba5-4c36-a300-cb6ede179087"). InnerVolumeSpecName "kube-api-access-s9rqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:43:28.621110 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621075    2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/33c1ec96-cba5-4c36-a300-cb6ede179087-local-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621110 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621107    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-token\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621120    2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-cacerts\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621132    2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-kubeconfig\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621145    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-dns-cert\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621158    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/33c1ec96-cba5-4c36-a300-cb6ede179087-istio-csr-ca-configmap\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.621296 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.621167    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9rqd\" (UniqueName: \"kubernetes.io/projected/33c1ec96-cba5-4c36-a300-cb6ede179087-kube-api-access-s9rqd\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:43:28.988655 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.988618    2561 generic.go:356] "Generic (PLEG): container finished" podID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerID="34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336" exitCode=0
Jul 30 16:43:28.988848 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.988665    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"
Jul 30 16:43:28.988848 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.988714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" event={"ID":"33c1ec96-cba5-4c36-a300-cb6ede179087","Type":"ContainerDied","Data":"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"}
Jul 30 16:43:28.988848 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.988796    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk" event={"ID":"33c1ec96-cba5-4c36-a300-cb6ede179087","Type":"ContainerDied","Data":"dadef078e4dca10a96ed638c6b05b8f6d50ea1add5a87272f780d0bd3ae58520"}
Jul 30 16:43:28.988848 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:28.988821    2561 scope.go:117] "RemoveContainer" containerID="34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"
Jul 30 16:43:29.012127 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.012103    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:29.022322 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.022299    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tpkwk"]
Jul 30 16:43:29.218586 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.218391    2561 scope.go:117] "RemoveContainer" containerID="34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"
Jul 30 16:43:29.218919 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:43:29.218806    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336\": container with ID starting with 34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336 not found: ID does not exist" containerID="34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"
Jul 30 16:43:29.218919 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.218839    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336"} err="failed to get container status \"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336\": rpc error: code = NotFound desc = could not find container \"34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336\": container with ID starting with 34ad4787f9502c323cad6086cd99180c78055eb92720ff73f4ecc60f440f7336 not found: ID does not exist"
Jul 30 16:43:29.341783 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.341752    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" path="/var/lib/kubelet/pods/33c1ec96-cba5-4c36-a300-cb6ede179087/volumes"
Jul 30 16:43:29.994828 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.994781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt" event={"ID":"41a88185-9b73-4109-ab1c-34da262e55de","Type":"ContainerStarted","Data":"0333bc5409bd545d5e63b30f95a343976aebb4ba01bbb531ecab874e9d5655ac"}
Jul 30 16:43:29.995114 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.994851    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:43:29.996179 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.996154    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4" event={"ID":"361f5299-762b-4cf5-8144-dd09e4c68bbd","Type":"ContainerStarted","Data":"c4efcc1c742b22dbfeda4471631337c3e0b90147d9222d63e60c1e2172316dcf"}
Jul 30 16:43:29.996297 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:29.996281    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:43:30.046231 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:30.046157    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt" podStartSLOduration=0.936072959 podStartE2EDuration="4.046143693s" totalImagesPullingTime="3.110070734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:30.045601818 +0000 UTC m=+439.303063239" watchObservedRunningTime="2026-07-30 16:43:30.046143693 +0000 UTC m=+439.303605113"
Jul 30 16:43:30.099279 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:30.099221    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4" podStartSLOduration=1.012649454 podStartE2EDuration="4.099204924s" totalImagesPullingTime="3.08655547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:30.08261703 +0000 UTC m=+439.340078449" watchObservedRunningTime="2026-07-30 16:43:30.099204924 +0000 UTC m=+439.356666344"
Jul 30 16:43:30.951005 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:30.950971    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tn6pp"
Jul 30 16:43:30.953185 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:43:30.953166    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-q9m9w"
Jul 30 16:44:01.002244 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:01.002214    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcgv4"
Jul 30 16:44:01.005352 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:01.005335    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bf2lt"
Jul 30 16:44:09.849292 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.849258    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:09.849721 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.849554    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerName="discovery"
Jul 30 16:44:09.849721 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.849565    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerName="discovery"
Jul 30 16:44:09.849721 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.849612    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c1ec96-cba5-4c36-a300-cb6ede179087" containerName="discovery"
Jul 30 16:44:09.853045 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.853018    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:09.855589 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.855562    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 16:44:09.855766 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.855660    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 16:44:09.856439 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.856412    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cwdw5\""
Jul 30 16:44:09.864879 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.864855    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:09.936399 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:09.936368    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvmnf\" (UniqueName: \"kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf\") pod \"kuadrant-operator-catalog-h2hqd\" (UID: \"c943a176-aba2-4934-a17a-a3ac71555c05\") " pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:10.037745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.037715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvmnf\" (UniqueName: \"kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf\") pod \"kuadrant-operator-catalog-h2hqd\" (UID: \"c943a176-aba2-4934-a17a-a3ac71555c05\") " pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:10.047389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.047364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvmnf\" (UniqueName: \"kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf\") pod \"kuadrant-operator-catalog-h2hqd\" (UID: \"c943a176-aba2-4934-a17a-a3ac71555c05\") " pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:10.159664 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.159599    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:10.159823 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.159812    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:10.285532 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.285465    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:10.287879 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:10.287851    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc943a176_aba2_4934_a17a_a3ac71555c05.slice/crio-25935efa958376f6ac15619893ba85b2bf54546e005cd0968d61d0199f9d131c WatchSource:0}: Error finding container 25935efa958376f6ac15619893ba85b2bf54546e005cd0968d61d0199f9d131c: Status 404 returned error can't find the container with id 25935efa958376f6ac15619893ba85b2bf54546e005cd0968d61d0199f9d131c
Jul 30 16:44:10.369107 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.369074    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p6s5l"]
Jul 30 16:44:10.372146 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.372132    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:10.380533 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.380511    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p6s5l"]
Jul 30 16:44:10.440601 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.440537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjfng\" (UniqueName: \"kubernetes.io/projected/d0a491a4-d01f-4c57-9fee-efce71c04b57-kube-api-access-bjfng\") pod \"kuadrant-operator-catalog-p6s5l\" (UID: \"d0a491a4-d01f-4c57-9fee-efce71c04b57\") " pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:10.541707 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.541682    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjfng\" (UniqueName: \"kubernetes.io/projected/d0a491a4-d01f-4c57-9fee-efce71c04b57-kube-api-access-bjfng\") pod \"kuadrant-operator-catalog-p6s5l\" (UID: \"d0a491a4-d01f-4c57-9fee-efce71c04b57\") " pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:10.550548 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.550529    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjfng\" (UniqueName: \"kubernetes.io/projected/d0a491a4-d01f-4c57-9fee-efce71c04b57-kube-api-access-bjfng\") pod \"kuadrant-operator-catalog-p6s5l\" (UID: \"d0a491a4-d01f-4c57-9fee-efce71c04b57\") " pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:10.681596 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.681563    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:10.804640 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:10.804619    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p6s5l"]
Jul 30 16:44:10.806736 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:10.806714    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a491a4_d01f_4c57_9fee_efce71c04b57.slice/crio-6c43cd4406b7bb632edfd47437a1f3d8f6ca70aefe4625fa5141d22e9c39ec07 WatchSource:0}: Error finding container 6c43cd4406b7bb632edfd47437a1f3d8f6ca70aefe4625fa5141d22e9c39ec07: Status 404 returned error can't find the container with id 6c43cd4406b7bb632edfd47437a1f3d8f6ca70aefe4625fa5141d22e9c39ec07
Jul 30 16:44:11.157199 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:11.157168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l" event={"ID":"d0a491a4-d01f-4c57-9fee-efce71c04b57","Type":"ContainerStarted","Data":"6c43cd4406b7bb632edfd47437a1f3d8f6ca70aefe4625fa5141d22e9c39ec07"}
Jul 30 16:44:11.158364 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:11.158333    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" event={"ID":"c943a176-aba2-4934-a17a-a3ac71555c05","Type":"ContainerStarted","Data":"25935efa958376f6ac15619893ba85b2bf54546e005cd0968d61d0199f9d131c"}
Jul 30 16:44:13.169050 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.169016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" event={"ID":"c943a176-aba2-4934-a17a-a3ac71555c05","Type":"ContainerStarted","Data":"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"}
Jul 30 16:44:13.169540 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.169073    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" podUID="c943a176-aba2-4934-a17a-a3ac71555c05" containerName="registry-server" containerID="cri-o://59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428" gracePeriod=2
Jul 30 16:44:13.170671 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.170642    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l" event={"ID":"d0a491a4-d01f-4c57-9fee-efce71c04b57","Type":"ContainerStarted","Data":"a73ab3715b3f752876dc0040756a0a8617a4bd2cb1e337c6218e6ecc0d42c913"}
Jul 30 16:44:13.193373 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.193332    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" podStartSLOduration=2.111643971 podStartE2EDuration="4.193318528s" totalImagesPullingTime="2.081674557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:13.192543547 +0000 UTC m=+482.450004993" watchObservedRunningTime="2026-07-30 16:44:13.193318528 +0000 UTC m=+482.450779948"
Jul 30 16:44:13.214538 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.214480    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l" podStartSLOduration=1.6611611800000001 podStartE2EDuration="3.214468704s" totalImagesPullingTime="1.553307524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:13.213568204 +0000 UTC m=+482.471029624" watchObservedRunningTime="2026-07-30 16:44:13.214468704 +0000 UTC m=+482.471930122"
Jul 30 16:44:13.405760 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.405734    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:13.465598 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.465459    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvmnf\" (UniqueName: \"kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf\") pod \"c943a176-aba2-4934-a17a-a3ac71555c05\" (UID: \"c943a176-aba2-4934-a17a-a3ac71555c05\") "
Jul 30 16:44:13.467511 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.467476    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf" (OuterVolumeSpecName: "kube-api-access-pvmnf") pod "c943a176-aba2-4934-a17a-a3ac71555c05" (UID: "c943a176-aba2-4934-a17a-a3ac71555c05"). InnerVolumeSpecName "kube-api-access-pvmnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:44:13.566605 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:13.566582    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvmnf\" (UniqueName: \"kubernetes.io/projected/c943a176-aba2-4934-a17a-a3ac71555c05-kube-api-access-pvmnf\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:44:14.175664 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.175627    2561 generic.go:356] "Generic (PLEG): container finished" podID="c943a176-aba2-4934-a17a-a3ac71555c05" containerID="59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428" exitCode=0
Jul 30 16:44:14.175664 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.175671    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd"
Jul 30 16:44:14.176175 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.175709    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" event={"ID":"c943a176-aba2-4934-a17a-a3ac71555c05","Type":"ContainerDied","Data":"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"}
Jul 30 16:44:14.176175 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.175765    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h2hqd" event={"ID":"c943a176-aba2-4934-a17a-a3ac71555c05","Type":"ContainerDied","Data":"25935efa958376f6ac15619893ba85b2bf54546e005cd0968d61d0199f9d131c"}
Jul 30 16:44:14.176175 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.175795    2561 scope.go:117] "RemoveContainer" containerID="59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"
Jul 30 16:44:14.184038 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.184019    2561 scope.go:117] "RemoveContainer" containerID="59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"
Jul 30 16:44:14.184308 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:44:14.184284    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428\": container with ID starting with 59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428 not found: ID does not exist" containerID="59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"
Jul 30 16:44:14.184415 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.184316    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428"} err="failed to get container status \"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428\": rpc error: code = NotFound desc = could not find container \"59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428\": container with ID starting with 59185995348b096a2132fbe59176298c9b347d2637b5a3c426a4bf01dbc4b428 not found: ID does not exist"
Jul 30 16:44:14.196912 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.196888    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:14.200349 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:14.200330    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h2hqd"]
Jul 30 16:44:15.339571 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:15.339538    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c943a176-aba2-4934-a17a-a3ac71555c05" path="/var/lib/kubelet/pods/c943a176-aba2-4934-a17a-a3ac71555c05/volumes"
Jul 30 16:44:20.682248 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:20.682215    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:20.682844 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:20.682273    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:20.703385 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:20.703363    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:21.225639 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:21.225612    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-p6s5l"
Jul 30 16:44:42.787854 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.787820    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"]
Jul 30 16:44:42.788238 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.788101    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c943a176-aba2-4934-a17a-a3ac71555c05" containerName="registry-server"
Jul 30 16:44:42.788238 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.788111    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c943a176-aba2-4934-a17a-a3ac71555c05" containerName="registry-server"
Jul 30 16:44:42.788238 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.788158    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c943a176-aba2-4934-a17a-a3ac71555c05" containerName="registry-server"
Jul 30 16:44:42.790087 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.790072    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:42.794564 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.794541    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-klffp\""
Jul 30 16:44:42.808034 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.808010    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"]
Jul 30 16:44:42.883536 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.883497    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g25v7\" (UniqueName: \"kubernetes.io/projected/6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6-kube-api-access-g25v7\") pod \"authorino-operator-75cc8d5cb7-wtfzz\" (UID: \"6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:42.984782 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:42.984742    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g25v7\" (UniqueName: \"kubernetes.io/projected/6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6-kube-api-access-g25v7\") pod \"authorino-operator-75cc8d5cb7-wtfzz\" (UID: \"6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:43.003506 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:43.003462    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g25v7\" (UniqueName: \"kubernetes.io/projected/6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6-kube-api-access-g25v7\") pod \"authorino-operator-75cc8d5cb7-wtfzz\" (UID: \"6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:43.099745 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:43.099718    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:43.249844 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:43.249822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"]
Jul 30 16:44:43.252663 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:43.252636    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b228f4b_2cd2_45e0_9e60_fcddd3b0dfd6.slice/crio-f3db8620da3ad3cac55d5147a68f2af27e4a16dd348bcb1edc0344a2b3cab80e WatchSource:0}: Error finding container f3db8620da3ad3cac55d5147a68f2af27e4a16dd348bcb1edc0344a2b3cab80e: Status 404 returned error can't find the container with id f3db8620da3ad3cac55d5147a68f2af27e4a16dd348bcb1edc0344a2b3cab80e
Jul 30 16:44:43.299201 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:43.299161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz" event={"ID":"6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6","Type":"ContainerStarted","Data":"f3db8620da3ad3cac55d5147a68f2af27e4a16dd348bcb1edc0344a2b3cab80e"}
Jul 30 16:44:45.957773 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:45.957741    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"]
Jul 30 16:44:45.959859 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:45.959842    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:45.962618 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:45.962597    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-v779g\""
Jul 30 16:44:45.962892 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:45.962874    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 16:44:45.987628 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:45.987604    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"]
Jul 30 16:44:46.112633 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.112599    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9plst\" (UniqueName: \"kubernetes.io/projected/1c45526a-02fa-4223-8153-f89ce872f074-kube-api-access-9plst\") pod \"dns-operator-controller-manager-864dd78c4c-tz7gs\" (UID: \"1c45526a-02fa-4223-8153-f89ce872f074\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:46.213401 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.213328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9plst\" (UniqueName: \"kubernetes.io/projected/1c45526a-02fa-4223-8153-f89ce872f074-kube-api-access-9plst\") pod \"dns-operator-controller-manager-864dd78c4c-tz7gs\" (UID: \"1c45526a-02fa-4223-8153-f89ce872f074\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:46.241136 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.241110    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9plst\" (UniqueName: \"kubernetes.io/projected/1c45526a-02fa-4223-8153-f89ce872f074-kube-api-access-9plst\") pod \"dns-operator-controller-manager-864dd78c4c-tz7gs\" (UID: \"1c45526a-02fa-4223-8153-f89ce872f074\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:46.270037 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.270016    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:46.321609 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.321555    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz" event={"ID":"6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6","Type":"ContainerStarted","Data":"b8cd36b9cdce6f27703ddb84028b0ab9d40be93bb56732ea42bd7c9201519f15"}
Jul 30 16:44:46.322535 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.322338    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:44:46.352798 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.352749    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz" podStartSLOduration=2.209250893 podStartE2EDuration="4.352731531s" totalImagesPullingTime="2.143480638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:46.351042238 +0000 UTC m=+515.608503661" watchObservedRunningTime="2026-07-30 16:44:46.352731531 +0000 UTC m=+515.610192951"
Jul 30 16:44:46.418461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:46.418437    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"]
Jul 30 16:44:46.419407 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:46.419381    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c45526a_02fa_4223_8153_f89ce872f074.slice/crio-0511caf0178ac61b725814385a941259c1b72273515414bba6c58bdf57719cc9 WatchSource:0}: Error finding container 0511caf0178ac61b725814385a941259c1b72273515414bba6c58bdf57719cc9: Status 404 returned error can't find the container with id 0511caf0178ac61b725814385a941259c1b72273515414bba6c58bdf57719cc9
Jul 30 16:44:47.327834 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:47.327798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs" event={"ID":"1c45526a-02fa-4223-8153-f89ce872f074","Type":"ContainerStarted","Data":"0511caf0178ac61b725814385a941259c1b72273515414bba6c58bdf57719cc9"}
Jul 30 16:44:48.333139 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:48.333106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs" event={"ID":"1c45526a-02fa-4223-8153-f89ce872f074","Type":"ContainerStarted","Data":"5307b73d628eb8f9001f38218dcb8630009cea46fb87e49183d4ea48e5fceabf"}
Jul 30 16:44:48.356975 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:48.356924    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs" podStartSLOduration=1.619936125 podStartE2EDuration="3.356909728s" totalImagesPullingTime="1.736973603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:48.355428357 +0000 UTC m=+517.612889799" watchObservedRunningTime="2026-07-30 16:44:48.356909728 +0000 UTC m=+517.614371148"
Jul 30 16:44:49.340562 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.340532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:44:49.409557 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.408813    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"]
Jul 30 16:44:49.413246 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.413205    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:49.416350 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.416329    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bflxd\""
Jul 30 16:44:49.427780 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.427756    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"]
Jul 30 16:44:49.541646 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.541575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm4z6\" (UniqueName: \"kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6\") pod \"limitador-operator-controller-manager-859d7b78bd-x8p4k\" (UID: \"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:49.642667 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.642629    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm4z6\" (UniqueName: \"kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6\") pod \"limitador-operator-controller-manager-859d7b78bd-x8p4k\" (UID: \"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:49.653817 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.653791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm4z6\" (UniqueName: \"kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6\") pod \"limitador-operator-controller-manager-859d7b78bd-x8p4k\" (UID: \"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:49.723756 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.723727    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:49.863328 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:49.863305    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"]
Jul 30 16:44:49.865374 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:49.865351    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbdbefa_78d7_4370_8bdc_dde21fa3b1b9.slice/crio-e7dea55e228977137572167d5c0e6824509b33bc407d1de6fa7cb7ba94d0af54 WatchSource:0}: Error finding container e7dea55e228977137572167d5c0e6824509b33bc407d1de6fa7cb7ba94d0af54: Status 404 returned error can't find the container with id e7dea55e228977137572167d5c0e6824509b33bc407d1de6fa7cb7ba94d0af54
Jul 30 16:44:50.342776 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:50.342740    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k" event={"ID":"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9","Type":"ContainerStarted","Data":"e7dea55e228977137572167d5c0e6824509b33bc407d1de6fa7cb7ba94d0af54"}
Jul 30 16:44:52.352475 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:52.352439    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k" event={"ID":"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9","Type":"ContainerStarted","Data":"e4a51a94109f11f51055e2db8a47e09b4cbf53a6e492e40f46a393ddfb1ed3cc"}
Jul 30 16:44:52.352922 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:52.352575    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:44:52.370980 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:52.370933    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k" podStartSLOduration=1.5354496210000002 podStartE2EDuration="3.370919862s" totalImagesPullingTime="1.835470241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:52.369647334 +0000 UTC m=+521.627108773" watchObservedRunningTime="2026-07-30 16:44:52.370919862 +0000 UTC m=+521.628381281"
Jul 30 16:44:54.300458 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.300423    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"]
Jul 30 16:44:54.303742 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.303726    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.307074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.307053    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dcm8j\""
Jul 30 16:44:54.320789 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.320765    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"]
Jul 30 16:44:54.380089 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.380063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.380089 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.380090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg24n\" (UniqueName: \"kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.481197 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.481166    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.481197 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.481196    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg24n\" (UniqueName: \"kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.481611 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.481593    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.491046 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.491025    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg24n\" (UniqueName: \"kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.613586 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.613555    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:44:54.757357 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:54.757330    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"]
Jul 30 16:44:54.758893 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:44:54.758871    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a14321_565f_4cc7_8418_073b1e51ccce.slice/crio-eba9187b7ae891d170bad8c95fddaf2f72cf86cac0c0864bf4937c0c5425d2f9 WatchSource:0}: Error finding container eba9187b7ae891d170bad8c95fddaf2f72cf86cac0c0864bf4937c0c5425d2f9: Status 404 returned error can't find the container with id eba9187b7ae891d170bad8c95fddaf2f72cf86cac0c0864bf4937c0c5425d2f9
Jul 30 16:44:55.366449 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:55.366415    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r" event={"ID":"a1a14321-565f-4cc7-8418-073b1e51ccce","Type":"ContainerStarted","Data":"eba9187b7ae891d170bad8c95fddaf2f72cf86cac0c0864bf4937c0c5425d2f9"}
Jul 30 16:44:58.335860 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:44:58.335831    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtfzz"
Jul 30 16:45:00.345704 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:00.345675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tz7gs"
Jul 30 16:45:00.397254 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:00.397222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r" event={"ID":"a1a14321-565f-4cc7-8418-073b1e51ccce","Type":"ContainerStarted","Data":"93a79054087e268d9fea218352bb2369670dfd4b8e74642cb6be8d7c09861921"}
Jul 30 16:45:00.397435 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:00.397355    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:45:00.418328 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:00.418276    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r" podStartSLOduration=0.96919022 podStartE2EDuration="6.418261123s" totalImagesPullingTime="5.449070903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:00.417418115 +0000 UTC m=+529.674879538" watchObservedRunningTime="2026-07-30 16:45:00.418261123 +0000 UTC m=+529.675722544"
Jul 30 16:45:03.359612 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:03.359531    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:45:11.404562 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:11.404533    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:45:12.680549 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.680518    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"]
Jul 30 16:45:12.684909 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.684890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.703368 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.703345    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"]
Jul 30 16:45:12.729158 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.729132    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.729269 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.729176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxv6d\" (UniqueName: \"kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.830095 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.830059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxv6d\" (UniqueName: \"kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.830288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.830124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.830422 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.830405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.845018 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.844992    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxv6d\" (UniqueName: \"kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d\") pod \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:12.994788 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:12.994713    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:13.134972 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.134950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"]
Jul 30 16:45:13.137953 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:13.137916    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e08418_4a7b_4df4_a2b2_35fdb2265c91.slice/crio-6609dd794265e382179ffee01c525e1ce49355c3d183b1be8dee99a931a0ffbe WatchSource:0}: Error finding container 6609dd794265e382179ffee01c525e1ce49355c3d183b1be8dee99a931a0ffbe: Status 404 returned error can't find the container with id 6609dd794265e382179ffee01c525e1ce49355c3d183b1be8dee99a931a0ffbe
Jul 30 16:45:13.262505 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.262423    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"]
Jul 30 16:45:13.270122 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.270096    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"]
Jul 30 16:45:13.281091 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.281066    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"]
Jul 30 16:45:13.281331 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.281292    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" containerName="manager" containerID="cri-o://93a79054087e268d9fea218352bb2369670dfd4b8e74642cb6be8d7c09861921" gracePeriod=2
Jul 30 16:45:13.290069 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.290007    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"]
Jul 30 16:45:13.296867 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.296838    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:13.297171 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297158    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" containerName="manager"
Jul 30 16:45:13.297246 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297173    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" containerName="manager"
Jul 30 16:45:13.297246 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297189    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" containerName="manager"
Jul 30 16:45:13.297246 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297195    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" containerName="manager"
Jul 30 16:45:13.297394 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297274    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" containerName="manager"
Jul 30 16:45:13.297394 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.297289    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" containerName="manager"
Jul 30 16:45:13.300119 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.300100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.302264 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:13.302243    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8lh8r\" is forbidden: User \"system:node:ip-10-0-132-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-133.ec2.internal' and this object" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:45:13.304859 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.304838    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"]
Jul 30 16:45:13.305006 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.304988    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k" podUID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" containerName="manager" containerID="cri-o://e4a51a94109f11f51055e2db8a47e09b4cbf53a6e492e40f46a393ddfb1ed3cc" gracePeriod=2
Jul 30 16:45:13.312832 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.312810    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:13.316016 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.315998    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.320401 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.320343    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:13.321529 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.321507    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"]
Jul 30 16:45:13.332585 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.332560    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:13.334084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.333968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.334084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nv29\" (UniqueName: \"kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.334084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnrfq\" (UniqueName: \"kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.334084 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334080    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"]
Jul 30 16:45:13.334313 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.334476 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334458    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" containerName="manager"
Jul 30 16:45:13.334550 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334480    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" containerName="manager"
Jul 30 16:45:13.334613 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.334600    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" containerName="manager"
Jul 30 16:45:13.337355 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.337338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:13.353870 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.353848    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"]
Jul 30 16:45:13.434542 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434515    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.434672 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nv29\" (UniqueName: \"kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.434734 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrfq\" (UniqueName: \"kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.434795 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434759    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.434853 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434811    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qmgr\" (UniqueName: \"kubernetes.io/projected/76f335cd-4c08-46de-afcc-7bd95e2dfa69-kube-api-access-8qmgr\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdbx\" (UID: \"76f335cd-4c08-46de-afcc-7bd95e2dfa69\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:13.434959 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.434935    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.435215 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.435196    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.444540 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.444503    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nv29\" (UniqueName: \"kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bkwgp\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.445531 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.445510    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrfq\" (UniqueName: \"kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq\") pod \"kuadrant-operator-controller-manager-55467d9f7-pjlnp\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.455520 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.455199    2561 generic.go:356] "Generic (PLEG): container finished" podID="a1a14321-565f-4cc7-8418-073b1e51ccce" containerID="93a79054087e268d9fea218352bb2369670dfd4b8e74642cb6be8d7c09861921" exitCode=0
Jul 30 16:45:13.457278 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:13.457253    2561 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc" container="manager"
Jul 30 16:45:13.458836 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.458811    2561 generic.go:356] "Generic (PLEG): container finished" podID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" containerID="e4a51a94109f11f51055e2db8a47e09b4cbf53a6e492e40f46a393ddfb1ed3cc" exitCode=0
Jul 30 16:45:13.529607 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.529579    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:45:13.535812 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.535793    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg24n\" (UniqueName: \"kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n\") pod \"a1a14321-565f-4cc7-8418-073b1e51ccce\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") "
Jul 30 16:45:13.535898 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.535862    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume\") pod \"a1a14321-565f-4cc7-8418-073b1e51ccce\" (UID: \"a1a14321-565f-4cc7-8418-073b1e51ccce\") "
Jul 30 16:45:13.535976 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.535962    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qmgr\" (UniqueName: \"kubernetes.io/projected/76f335cd-4c08-46de-afcc-7bd95e2dfa69-kube-api-access-8qmgr\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdbx\" (UID: \"76f335cd-4c08-46de-afcc-7bd95e2dfa69\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:13.536249 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.536212    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a1a14321-565f-4cc7-8418-073b1e51ccce" (UID: "a1a14321-565f-4cc7-8418-073b1e51ccce"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:45:13.536663 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.536649    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:45:13.537789 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.537768    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n" (OuterVolumeSpecName: "kube-api-access-lg24n") pod "a1a14321-565f-4cc7-8418-073b1e51ccce" (UID: "a1a14321-565f-4cc7-8418-073b1e51ccce"). InnerVolumeSpecName "kube-api-access-lg24n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:45:13.545095 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.545078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qmgr\" (UniqueName: \"kubernetes.io/projected/76f335cd-4c08-46de-afcc-7bd95e2dfa69-kube-api-access-8qmgr\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdbx\" (UID: \"76f335cd-4c08-46de-afcc-7bd95e2dfa69\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:13.637128 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.637104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm4z6\" (UniqueName: \"kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6\") pod \"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9\" (UID: \"9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9\") "
Jul 30 16:45:13.637247 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.637212    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1a14321-565f-4cc7-8418-073b1e51ccce-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:13.637247 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.637223    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg24n\" (UniqueName: \"kubernetes.io/projected/a1a14321-565f-4cc7-8418-073b1e51ccce-kube-api-access-lg24n\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:13.638918 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.638896    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6" (OuterVolumeSpecName: "kube-api-access-xm4z6") pod "9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" (UID: "9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9"). InnerVolumeSpecName "kube-api-access-xm4z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:45:13.700711 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.700684    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:13.707737 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.707713    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:13.713724 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.713703    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:13.738012 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.737982    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm4z6\" (UniqueName: \"kubernetes.io/projected/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9-kube-api-access-xm4z6\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:13.852649 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.852443    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:13.856597 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:13.856337    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1408413c_8c5f_44ba_96d4_625781475722.slice/crio-69d084049ad1886996c6b68fd88655673bf48a6ec2f50fa1706e314b5d1db834 WatchSource:0}: Error finding container 69d084049ad1886996c6b68fd88655673bf48a6ec2f50fa1706e314b5d1db834: Status 404 returned error can't find the container with id 69d084049ad1886996c6b68fd88655673bf48a6ec2f50fa1706e314b5d1db834
Jul 30 16:45:13.874341 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.874217    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:13.878089 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:13.878067    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed80a03_9ac4_4e6e_adcf_159e48dac111.slice/crio-b7078eb1ee37a7e048b661919b35701a7d5eab2be3ffb284320531b25669c548 WatchSource:0}: Error finding container b7078eb1ee37a7e048b661919b35701a7d5eab2be3ffb284320531b25669c548: Status 404 returned error can't find the container with id b7078eb1ee37a7e048b661919b35701a7d5eab2be3ffb284320531b25669c548
Jul 30 16:45:13.894750 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:13.894726    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"]
Jul 30 16:45:13.897710 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:13.897683    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f335cd_4c08_46de_afcc_7bd95e2dfa69.slice/crio-94923e145cd4cf923ee483c382d8f96718d9ae506d5fca674e83b1332a3ceb6d WatchSource:0}: Error finding container 94923e145cd4cf923ee483c382d8f96718d9ae506d5fca674e83b1332a3ceb6d: Status 404 returned error can't find the container with id 94923e145cd4cf923ee483c382d8f96718d9ae506d5fca674e83b1332a3ceb6d
Jul 30 16:45:14.467479 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.467439    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" event={"ID":"1408413c-8c5f-44ba-96d4-625781475722","Type":"ContainerStarted","Data":"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"}
Jul 30 16:45:14.467701 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.467517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" event={"ID":"1408413c-8c5f-44ba-96d4-625781475722","Type":"ContainerStarted","Data":"69d084049ad1886996c6b68fd88655673bf48a6ec2f50fa1706e314b5d1db834"}
Jul 30 16:45:14.467701 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.467573    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:14.469199 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.469167    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" containerName="manager" containerID="cri-o://b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75" gracePeriod=2
Jul 30 16:45:14.470515 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.470475    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x8p4k"
Jul 30 16:45:14.470594 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.470541    2561 scope.go:117] "RemoveContainer" containerID="e4a51a94109f11f51055e2db8a47e09b4cbf53a6e492e40f46a393ddfb1ed3cc"
Jul 30 16:45:14.471929 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.471912    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8lh8r"
Jul 30 16:45:14.474372 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.473937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" event={"ID":"fed80a03-9ac4-4e6e-adcf-159e48dac111","Type":"ContainerStarted","Data":"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"}
Jul 30 16:45:14.474372 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.473979    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" event={"ID":"fed80a03-9ac4-4e6e-adcf-159e48dac111","Type":"ContainerStarted","Data":"b7078eb1ee37a7e048b661919b35701a7d5eab2be3ffb284320531b25669c548"}
Jul 30 16:45:14.474372 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.474076    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:14.475948 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.475927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx" event={"ID":"76f335cd-4c08-46de-afcc-7bd95e2dfa69","Type":"ContainerStarted","Data":"20abd52b7783d7e84b48a3ecaab72ced11727fd42d5be595f0ca8c8ab4d0f811"}
Jul 30 16:45:14.476058 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.475974    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:14.476058 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.475998    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx" event={"ID":"76f335cd-4c08-46de-afcc-7bd95e2dfa69","Type":"ContainerStarted","Data":"94923e145cd4cf923ee483c382d8f96718d9ae506d5fca674e83b1332a3ceb6d"}
Jul 30 16:45:14.480732 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.480604    2561 scope.go:117] "RemoveContainer" containerID="93a79054087e268d9fea218352bb2369670dfd4b8e74642cb6be8d7c09861921"
Jul 30 16:45:14.490329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.490289    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" podStartSLOduration=1.490276542 podStartE2EDuration="1.490276542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:14.4892161 +0000 UTC m=+543.746677544" watchObservedRunningTime="2026-07-30 16:45:14.490276542 +0000 UTC m=+543.747737961"
Jul 30 16:45:14.491724 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:14.491698    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" is forbidden: User \"system:node:ip-10-0-132-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-133.ec2.internal' and this object" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:14.518178 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.518135    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" podStartSLOduration=1.51812176 podStartE2EDuration="1.51812176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:14.516330826 +0000 UTC m=+543.773792246" watchObservedRunningTime="2026-07-30 16:45:14.51812176 +0000 UTC m=+543.775583180"
Jul 30 16:45:14.537549 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.537511    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx" podStartSLOduration=1.5374977319999998 podStartE2EDuration="1.537497732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:14.535661624 +0000 UTC m=+543.793123043" watchObservedRunningTime="2026-07-30 16:45:14.537497732 +0000 UTC m=+543.794959142"
Jul 30 16:45:14.703142 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.703117    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:14.705300 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:14.705277    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" is forbidden: User \"system:node:ip-10-0-132-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-133.ec2.internal' and this object" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:14.746465 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.746403    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxv6d\" (UniqueName: \"kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d\") pod \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") "
Jul 30 16:45:14.746465 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.746436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume\") pod \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\" (UID: \"d7e08418-4a7b-4df4-a2b2-35fdb2265c91\") "
Jul 30 16:45:14.746741 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.746722    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d7e08418-4a7b-4df4-a2b2-35fdb2265c91" (UID: "d7e08418-4a7b-4df4-a2b2-35fdb2265c91"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:45:14.748315 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.748292    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d" (OuterVolumeSpecName: "kube-api-access-bxv6d") pod "d7e08418-4a7b-4df4-a2b2-35fdb2265c91" (UID: "d7e08418-4a7b-4df4-a2b2-35fdb2265c91"). InnerVolumeSpecName "kube-api-access-bxv6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:45:14.847113 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.847081    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxv6d\" (UniqueName: \"kubernetes.io/projected/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-kube-api-access-bxv6d\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:14.847113 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:14.847106    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7e08418-4a7b-4df4-a2b2-35fdb2265c91-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:15.340668 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.340639    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9" path="/var/lib/kubelet/pods/9dbdbefa-78d7-4370-8bdc-dde21fa3b1b9/volumes"
Jul 30 16:45:15.340975 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.340963    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a14321-565f-4cc7-8418-073b1e51ccce" path="/var/lib/kubelet/pods/a1a14321-565f-4cc7-8418-073b1e51ccce/volumes"
Jul 30 16:45:15.341258 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.341247    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" path="/var/lib/kubelet/pods/d7e08418-4a7b-4df4-a2b2-35fdb2265c91/volumes"
Jul 30 16:45:15.481417 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.481383    2561 generic.go:356] "Generic (PLEG): container finished" podID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" containerID="b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75" exitCode=2
Jul 30 16:45:15.481611 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.481428    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:15.481611 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.481502    2561 scope.go:117] "RemoveContainer" containerID="b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75"
Jul 30 16:45:15.486132 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:15.486105    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2t4nc\" is forbidden: User \"system:node:ip-10-0-132-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-133.ec2.internal' and this object" podUID="d7e08418-4a7b-4df4-a2b2-35fdb2265c91" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2t4nc"
Jul 30 16:45:15.490275 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.490252    2561 scope.go:117] "RemoveContainer" containerID="b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75"
Jul 30 16:45:15.490598 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:15.490577    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75\": container with ID starting with b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75 not found: ID does not exist" containerID="b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75"
Jul 30 16:45:15.490708 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:15.490607    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75"} err="failed to get container status \"b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75\": rpc error: code = NotFound desc = could not find container \"b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75\": container with ID starting with b3d5d3d615ca88110406c9d93a9fc98a869c7da0b2589ad972ed9216d0a0fd75 not found: ID does not exist"
Jul 30 16:45:25.485907 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.485875    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:25.486461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.485946    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdbx"
Jul 30 16:45:25.486461 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.486236    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:25.598049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.598018    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:25.598245 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.598222    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" podUID="1408413c-8c5f-44ba-96d4-625781475722" containerName="manager" containerID="cri-o://0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132" gracePeriod=10
Jul 30 16:45:25.840038 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.840016    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:25.847909 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.847885    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 16:45:25.848208 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.848196    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1408413c-8c5f-44ba-96d4-625781475722" containerName="manager"
Jul 30 16:45:25.848249 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.848209    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1408413c-8c5f-44ba-96d4-625781475722" containerName="manager"
Jul 30 16:45:25.848292 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.848282    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1408413c-8c5f-44ba-96d4-625781475722" containerName="manager"
Jul 30 16:45:25.850127 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.850110    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:25.885770 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.885744    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 16:45:25.940422 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.940397    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nv29\" (UniqueName: \"kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29\") pod \"1408413c-8c5f-44ba-96d4-625781475722\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") "
Jul 30 16:45:25.940563 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.940499    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume\") pod \"1408413c-8c5f-44ba-96d4-625781475722\" (UID: \"1408413c-8c5f-44ba-96d4-625781475722\") "
Jul 30 16:45:25.940667 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.940648    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:25.940739 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.940723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djtbd\" (UniqueName: \"kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:25.940858 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.940838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1408413c-8c5f-44ba-96d4-625781475722" (UID: "1408413c-8c5f-44ba-96d4-625781475722"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:45:25.942256 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:25.942235    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29" (OuterVolumeSpecName: "kube-api-access-2nv29") pod "1408413c-8c5f-44ba-96d4-625781475722" (UID: "1408413c-8c5f-44ba-96d4-625781475722"). InnerVolumeSpecName "kube-api-access-2nv29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:45:26.041964 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.041899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djtbd\" (UniqueName: \"kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.041964 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.041953    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.042123 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.041984    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1408413c-8c5f-44ba-96d4-625781475722-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:26.042123 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.041995    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nv29\" (UniqueName: \"kubernetes.io/projected/1408413c-8c5f-44ba-96d4-625781475722-kube-api-access-2nv29\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:26.042286 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.042271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.053883 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.053861    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djtbd\" (UniqueName: \"kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd\") pod \"kuadrant-operator-controller-manager-86878c769f-bhlpb\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.159119 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.159100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.294820 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.294799    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 16:45:26.297420 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:26.297392    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03fce07_5432_4d60_a584_78f7e53c7768.slice/crio-389deeeaf391c3f767c716ce2923156a98314801786c0d6a6b7ef3256c3c4155 WatchSource:0}: Error finding container 389deeeaf391c3f767c716ce2923156a98314801786c0d6a6b7ef3256c3c4155: Status 404 returned error can't find the container with id 389deeeaf391c3f767c716ce2923156a98314801786c0d6a6b7ef3256c3c4155
Jul 30 16:45:26.542991 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.542959    2561 generic.go:356] "Generic (PLEG): container finished" podID="1408413c-8c5f-44ba-96d4-625781475722" containerID="0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132" exitCode=0
Jul 30 16:45:26.543468 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.543008    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"
Jul 30 16:45:26.543468 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.543050    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" event={"ID":"1408413c-8c5f-44ba-96d4-625781475722","Type":"ContainerDied","Data":"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"}
Jul 30 16:45:26.543468 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.543142    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp" event={"ID":"1408413c-8c5f-44ba-96d4-625781475722","Type":"ContainerDied","Data":"69d084049ad1886996c6b68fd88655673bf48a6ec2f50fa1706e314b5d1db834"}
Jul 30 16:45:26.543468 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.543175    2561 scope.go:117] "RemoveContainer" containerID="0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"
Jul 30 16:45:26.544808 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.544750    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" event={"ID":"e03fce07-5432-4d60-a584-78f7e53c7768","Type":"ContainerStarted","Data":"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"}
Jul 30 16:45:26.544808 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.544786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" event={"ID":"e03fce07-5432-4d60-a584-78f7e53c7768","Type":"ContainerStarted","Data":"389deeeaf391c3f767c716ce2923156a98314801786c0d6a6b7ef3256c3c4155"}
Jul 30 16:45:26.544977 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.544877    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:26.552280 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.552265    2561 scope.go:117] "RemoveContainer" containerID="0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"
Jul 30 16:45:26.552541 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:26.552523    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132\": container with ID starting with 0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132 not found: ID does not exist" containerID="0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"
Jul 30 16:45:26.552586 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.552548    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132"} err="failed to get container status \"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132\": rpc error: code = NotFound desc = could not find container \"0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132\": container with ID starting with 0d6ab1dd30173a4d21c098827fc4b37217d9faaa20cd9d07d15857b3480ac132 not found: ID does not exist"
Jul 30 16:45:26.572973 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.572935    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" podStartSLOduration=1.5729240899999999 podStartE2EDuration="1.57292409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:26.572107706 +0000 UTC m=+555.829569159" watchObservedRunningTime="2026-07-30 16:45:26.57292409 +0000 UTC m=+555.830385509"
Jul 30 16:45:26.587200 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.587174    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:26.591885 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:26.591862    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bkwgp"]
Jul 30 16:45:27.339927 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:27.339898    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1408413c-8c5f-44ba-96d4-625781475722" path="/var/lib/kubelet/pods/1408413c-8c5f-44ba-96d4-625781475722/volumes"
Jul 30 16:45:37.552632 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.552603    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 16:45:37.607503 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.607465    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:37.607719 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.607681    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" podUID="fed80a03-9ac4-4e6e-adcf-159e48dac111" containerName="manager" containerID="cri-o://0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435" gracePeriod=10
Jul 30 16:45:37.859786 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.859760    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:37.935428 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.935397    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume\") pod \"fed80a03-9ac4-4e6e-adcf-159e48dac111\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") "
Jul 30 16:45:37.935608 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.935461    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnrfq\" (UniqueName: \"kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq\") pod \"fed80a03-9ac4-4e6e-adcf-159e48dac111\" (UID: \"fed80a03-9ac4-4e6e-adcf-159e48dac111\") "
Jul 30 16:45:37.935842 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.935817    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fed80a03-9ac4-4e6e-adcf-159e48dac111" (UID: "fed80a03-9ac4-4e6e-adcf-159e48dac111"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:45:37.937516 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:37.937476    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq" (OuterVolumeSpecName: "kube-api-access-lnrfq") pod "fed80a03-9ac4-4e6e-adcf-159e48dac111" (UID: "fed80a03-9ac4-4e6e-adcf-159e48dac111"). InnerVolumeSpecName "kube-api-access-lnrfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:45:38.036983 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.036940    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fed80a03-9ac4-4e6e-adcf-159e48dac111-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:38.036983 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.036980    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnrfq\" (UniqueName: \"kubernetes.io/projected/fed80a03-9ac4-4e6e-adcf-159e48dac111-kube-api-access-lnrfq\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:45:38.607010 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.606978    2561 generic.go:356] "Generic (PLEG): container finished" podID="fed80a03-9ac4-4e6e-adcf-159e48dac111" containerID="0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435" exitCode=0
Jul 30 16:45:38.607430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.607017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" event={"ID":"fed80a03-9ac4-4e6e-adcf-159e48dac111","Type":"ContainerDied","Data":"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"}
Jul 30 16:45:38.607430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.607064    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp" event={"ID":"fed80a03-9ac4-4e6e-adcf-159e48dac111","Type":"ContainerDied","Data":"b7078eb1ee37a7e048b661919b35701a7d5eab2be3ffb284320531b25669c548"}
Jul 30 16:45:38.607430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.607085    2561 scope.go:117] "RemoveContainer" containerID="0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"
Jul 30 16:45:38.607430 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.607020    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"
Jul 30 16:45:38.616411 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.616396    2561 scope.go:117] "RemoveContainer" containerID="0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"
Jul 30 16:45:38.616672 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:45:38.616655    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435\": container with ID starting with 0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435 not found: ID does not exist" containerID="0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"
Jul 30 16:45:38.616729 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.616680    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435"} err="failed to get container status \"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435\": rpc error: code = NotFound desc = could not find container \"0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435\": container with ID starting with 0de6fa78531f33377c28c000ca8ccdf0a9b9cb28f7ac1c8770f5e7e6547d5435 not found: ID does not exist"
Jul 30 16:45:38.635052 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.633423    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:38.639926 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:38.639903    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pjlnp"]
Jul 30 16:45:39.340117 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:39.340079    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed80a03-9ac4-4e6e-adcf-159e48dac111" path="/var/lib/kubelet/pods/fed80a03-9ac4-4e6e-adcf-159e48dac111/volumes"
Jul 30 16:45:55.645967 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.645930    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:45:55.646352 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.646239    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed80a03-9ac4-4e6e-adcf-159e48dac111" containerName="manager"
Jul 30 16:45:55.646352 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.646250    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed80a03-9ac4-4e6e-adcf-159e48dac111" containerName="manager"
Jul 30 16:45:55.646352 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.646319    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed80a03-9ac4-4e6e-adcf-159e48dac111" containerName="manager"
Jul 30 16:45:55.648954 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.648936    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.653021 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.652999    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pvr8r\""
Jul 30 16:45:55.653109 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.653056    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 16:45:55.659284 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.659255    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:45:55.692702 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.692675    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:45:55.798683 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.798658    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/919c174f-f968-4953-b5c6-c168c512f43d-config-file\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.798825 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.798692    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626g9\" (UniqueName: \"kubernetes.io/projected/919c174f-f968-4953-b5c6-c168c512f43d-kube-api-access-626g9\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.899176 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.899108    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/919c174f-f968-4953-b5c6-c168c512f43d-config-file\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.899176 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.899142    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626g9\" (UniqueName: \"kubernetes.io/projected/919c174f-f968-4953-b5c6-c168c512f43d-kube-api-access-626g9\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.899730 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.899711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/919c174f-f968-4953-b5c6-c168c512f43d-config-file\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.907369 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.907347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-626g9\" (UniqueName: \"kubernetes.io/projected/919c174f-f968-4953-b5c6-c168c512f43d-kube-api-access-626g9\") pod \"limitador-limitador-54d6d988d5-rclnf\" (UID: \"919c174f-f968-4953-b5c6-c168c512f43d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:55.959225 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:55.959198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:45:56.060389 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.060290    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:45:56.063428 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.063404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:45:56.065888 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.065864    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-p8bft\""
Jul 30 16:45:56.077282 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.077261    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:45:56.091575 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.091550    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:45:56.093743 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:56.093720    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919c174f_f968_4953_b5c6_c168c512f43d.slice/crio-b60c181803fc9b8848a12b3324a02bda24397b611dc2e6204bd76b1725567be3 WatchSource:0}: Error finding container b60c181803fc9b8848a12b3324a02bda24397b611dc2e6204bd76b1725567be3: Status 404 returned error can't find the container with id b60c181803fc9b8848a12b3324a02bda24397b611dc2e6204bd76b1725567be3
Jul 30 16:45:56.202349 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.202285    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xvm7\" (UniqueName: \"kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7\") pod \"authorino-58cd955675-s89sb\" (UID: \"a70a9176-e58a-478f-b4f6-bce2d91c6817\") " pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:45:56.303093 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.303062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xvm7\" (UniqueName: \"kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7\") pod \"authorino-58cd955675-s89sb\" (UID: \"a70a9176-e58a-478f-b4f6-bce2d91c6817\") " pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:45:56.320846 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.320819    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xvm7\" (UniqueName: \"kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7\") pod \"authorino-58cd955675-s89sb\" (UID: \"a70a9176-e58a-478f-b4f6-bce2d91c6817\") " pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:45:56.374280 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.374251    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:45:56.503809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.503762    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:45:56.506338 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:45:56.506314    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda70a9176_e58a_478f_b4f6_bce2d91c6817.slice/crio-854e81f20b81a9efb97c6d5825746cf4c07f576f71b6620f67b298742bc7e983 WatchSource:0}: Error finding container 854e81f20b81a9efb97c6d5825746cf4c07f576f71b6620f67b298742bc7e983: Status 404 returned error can't find the container with id 854e81f20b81a9efb97c6d5825746cf4c07f576f71b6620f67b298742bc7e983
Jul 30 16:45:56.697891 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.697853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s89sb" event={"ID":"a70a9176-e58a-478f-b4f6-bce2d91c6817","Type":"ContainerStarted","Data":"854e81f20b81a9efb97c6d5825746cf4c07f576f71b6620f67b298742bc7e983"}
Jul 30 16:45:56.699353 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:45:56.699328    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf" event={"ID":"919c174f-f968-4953-b5c6-c168c512f43d","Type":"ContainerStarted","Data":"b60c181803fc9b8848a12b3324a02bda24397b611dc2e6204bd76b1725567be3"}
Jul 30 16:46:00.467181 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.467143    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:46:00.745228 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.745148    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf" event={"ID":"919c174f-f968-4953-b5c6-c168c512f43d","Type":"ContainerStarted","Data":"bf504fec7aa83653188c1e28bdacb5f10c35ec1b21039f8d6bfc7870d6bb4ea4"}
Jul 30 16:46:00.745392 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.745308    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:46:00.746637 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.746617    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s89sb" event={"ID":"a70a9176-e58a-478f-b4f6-bce2d91c6817","Type":"ContainerStarted","Data":"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"}
Jul 30 16:46:00.768055 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.768017    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf" podStartSLOduration=1.501919408 podStartE2EDuration="5.768005373s" totalImagesPullingTime="4.266085965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:00.765977763 +0000 UTC m=+590.023439183" watchObservedRunningTime="2026-07-30 16:46:00.768005373 +0000 UTC m=+590.025466792"
Jul 30 16:46:00.785824 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:00.785785    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s89sb" podStartSLOduration=0.929749571 podStartE2EDuration="4.785772493s" totalImagesPullingTime="3.856022922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:00.784985265 +0000 UTC m=+590.042446686" watchObservedRunningTime="2026-07-30 16:46:00.785772493 +0000 UTC m=+590.043233913"
Jul 30 16:46:01.751279 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:01.751230    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s89sb" podUID="a70a9176-e58a-478f-b4f6-bce2d91c6817" containerName="authorino" containerID="cri-o://21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa" gracePeriod=30
Jul 30 16:46:01.997217 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:01.997195    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:46:02.057120 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.057043    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xvm7\" (UniqueName: \"kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7\") pod \"a70a9176-e58a-478f-b4f6-bce2d91c6817\" (UID: \"a70a9176-e58a-478f-b4f6-bce2d91c6817\") "
Jul 30 16:46:02.059024 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.058993    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7" (OuterVolumeSpecName: "kube-api-access-5xvm7") pod "a70a9176-e58a-478f-b4f6-bce2d91c6817" (UID: "a70a9176-e58a-478f-b4f6-bce2d91c6817"). InnerVolumeSpecName "kube-api-access-5xvm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:02.158394 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.158362    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xvm7\" (UniqueName: \"kubernetes.io/projected/a70a9176-e58a-478f-b4f6-bce2d91c6817-kube-api-access-5xvm7\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:46:02.757340 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.757310    2561 generic.go:356] "Generic (PLEG): container finished" podID="a70a9176-e58a-478f-b4f6-bce2d91c6817" containerID="21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa" exitCode=0
Jul 30 16:46:02.757864 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.757361    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s89sb"
Jul 30 16:46:02.757864 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.757405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s89sb" event={"ID":"a70a9176-e58a-478f-b4f6-bce2d91c6817","Type":"ContainerDied","Data":"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"}
Jul 30 16:46:02.757864 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.757448    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s89sb" event={"ID":"a70a9176-e58a-478f-b4f6-bce2d91c6817","Type":"ContainerDied","Data":"854e81f20b81a9efb97c6d5825746cf4c07f576f71b6620f67b298742bc7e983"}
Jul 30 16:46:02.757864 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.757466    2561 scope.go:117] "RemoveContainer" containerID="21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"
Jul 30 16:46:02.765818 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.765804    2561 scope.go:117] "RemoveContainer" containerID="21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"
Jul 30 16:46:02.766089 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:46:02.766072    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa\": container with ID starting with 21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa not found: ID does not exist" containerID="21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"
Jul 30 16:46:02.766137 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.766097    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa"} err="failed to get container status \"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa\": rpc error: code = NotFound desc = could not find container \"21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa\": container with ID starting with 21193a14158192322c4f5768bfdf8880b9c2926edf0ab255d86d61a60f18d8aa not found: ID does not exist"
Jul 30 16:46:02.782505 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.779914    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:46:02.784663 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:02.783455    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s89sb"]
Jul 30 16:46:03.340866 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:03.340834    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a70a9176-e58a-478f-b4f6-bce2d91c6817" path="/var/lib/kubelet/pods/a70a9176-e58a-478f-b4f6-bce2d91c6817/volumes"
Jul 30 16:46:11.152311 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:11.152282    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:46:11.152761 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:11.152390    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:46:11.157730 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:11.157712    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:46:11.158067 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:11.158051    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:46:11.752816 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:11.752788    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rclnf"
Jul 30 16:46:28.104230 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.104195    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:28.104647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.104548    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a70a9176-e58a-478f-b4f6-bce2d91c6817" containerName="authorino"
Jul 30 16:46:28.104647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.104559    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a70a9176-e58a-478f-b4f6-bce2d91c6817" containerName="authorino"
Jul 30 16:46:28.104647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.104616    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a70a9176-e58a-478f-b4f6-bce2d91c6817" containerName="authorino"
Jul 30 16:46:28.110130 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.110109    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:28.112659 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.112635    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-p8bft\""
Jul 30 16:46:28.114385 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.114359    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:28.155610 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.155583    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlgdb\" (UniqueName: \"kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb\") pod \"authorino-5d4b67c446-8lngw\" (UID: \"b7527a73-5c86-476f-a214-80420923d2dd\") " pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:28.256541 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.256479    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlgdb\" (UniqueName: \"kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb\") pod \"authorino-5d4b67c446-8lngw\" (UID: \"b7527a73-5c86-476f-a214-80420923d2dd\") " pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:28.265074 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.265039    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlgdb\" (UniqueName: \"kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb\") pod \"authorino-5d4b67c446-8lngw\" (UID: \"b7527a73-5c86-476f-a214-80420923d2dd\") " pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:28.339576 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.339548    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:28.339773 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.339761    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:28.375871 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.375806    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:28.378312 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.378268    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:28.394392 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.394349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:28.459118 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.459087    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlg2c\" (UniqueName: \"kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c\") pod \"authorino-78fd44b96b-g74vt\" (UID: \"9f8b14e8-0f27-4462-a22c-b547eed14964\") " pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:28.470254 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.470225    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:28.473192 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:46:28.473167    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7527a73_5c86_476f_a214_80420923d2dd.slice/crio-f29ff83a277880495971b8436f57baf6b68b7d109ea721fc0b730ad2440bb37e WatchSource:0}: Error finding container f29ff83a277880495971b8436f57baf6b68b7d109ea721fc0b730ad2440bb37e: Status 404 returned error can't find the container with id f29ff83a277880495971b8436f57baf6b68b7d109ea721fc0b730ad2440bb37e
Jul 30 16:46:28.560126 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.560094    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlg2c\" (UniqueName: \"kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c\") pod \"authorino-78fd44b96b-g74vt\" (UID: \"9f8b14e8-0f27-4462-a22c-b547eed14964\") " pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:28.568956 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.568932    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlg2c\" (UniqueName: \"kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c\") pod \"authorino-78fd44b96b-g74vt\" (UID: \"9f8b14e8-0f27-4462-a22c-b547eed14964\") " pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:28.573404 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.573379    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:28.573661 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.573646    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:28.603596 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.603569    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:46:28.606677 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.606659    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.609570 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.609544    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 16:46:28.616270 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.616247    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:46:28.661147 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.661119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8cg2\" (UniqueName: \"kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.661293 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.661247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.701933 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.701898    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:28.704976 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:46:28.704950    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f8b14e8_0f27_4462_a22c_b547eed14964.slice/crio-8053174e9f4b2b28ed61a39d942fc6e1e2bab6d2f71f2651f39590c59e9b5f23 WatchSource:0}: Error finding container 8053174e9f4b2b28ed61a39d942fc6e1e2bab6d2f71f2651f39590c59e9b5f23: Status 404 returned error can't find the container with id 8053174e9f4b2b28ed61a39d942fc6e1e2bab6d2f71f2651f39590c59e9b5f23
Jul 30 16:46:28.761656 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.761632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8cg2\" (UniqueName: \"kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.761763 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.761687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.763895 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.763879    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.769773 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.769752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8cg2\" (UniqueName: \"kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2\") pod \"authorino-7bb86cb758-5djx9\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") " pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:28.872468 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.872438    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78fd44b96b-g74vt" event={"ID":"9f8b14e8-0f27-4462-a22c-b547eed14964","Type":"ContainerStarted","Data":"8053174e9f4b2b28ed61a39d942fc6e1e2bab6d2f71f2651f39590c59e9b5f23"}
Jul 30 16:46:28.873609 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.873586    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8lngw" event={"ID":"b7527a73-5c86-476f-a214-80420923d2dd","Type":"ContainerStarted","Data":"f29ff83a277880495971b8436f57baf6b68b7d109ea721fc0b730ad2440bb37e"}
Jul 30 16:46:28.917740 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:28.917720    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:46:29.056753 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.056669    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:46:29.059334 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:46:29.059301    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01777f05_1e9e_4327_b002_702fc73384b6.slice/crio-86c1a901bc783ed5ce5afdf95625bbb55bcc8f2d431c130905c4f7970db9d554 WatchSource:0}: Error finding container 86c1a901bc783ed5ce5afdf95625bbb55bcc8f2d431c130905c4f7970db9d554: Status 404 returned error can't find the container with id 86c1a901bc783ed5ce5afdf95625bbb55bcc8f2d431c130905c4f7970db9d554
Jul 30 16:46:29.879105 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.879070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb86cb758-5djx9" event={"ID":"01777f05-1e9e-4327-b002-702fc73384b6","Type":"ContainerStarted","Data":"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"}
Jul 30 16:46:29.879632 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.879121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb86cb758-5djx9" event={"ID":"01777f05-1e9e-4327-b002-702fc73384b6","Type":"ContainerStarted","Data":"86c1a901bc783ed5ce5afdf95625bbb55bcc8f2d431c130905c4f7970db9d554"}
Jul 30 16:46:29.880616 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.880590    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78fd44b96b-g74vt" event={"ID":"9f8b14e8-0f27-4462-a22c-b547eed14964","Type":"ContainerStarted","Data":"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"}
Jul 30 16:46:29.880748 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.880610    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-78fd44b96b-g74vt" podUID="9f8b14e8-0f27-4462-a22c-b547eed14964" containerName="authorino" containerID="cri-o://0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c" gracePeriod=30
Jul 30 16:46:29.882123 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.882102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8lngw" event={"ID":"b7527a73-5c86-476f-a214-80420923d2dd","Type":"ContainerStarted","Data":"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"}
Jul 30 16:46:29.882214 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.882145    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-8lngw" podUID="b7527a73-5c86-476f-a214-80420923d2dd" containerName="authorino" containerID="cri-o://46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa" gracePeriod=30
Jul 30 16:46:29.895679 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.895642    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bb86cb758-5djx9" podStartSLOduration=1.597439793 podStartE2EDuration="1.895629344s" totalImagesPullingTime="298.189551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:29.894592728 +0000 UTC m=+619.152054148" watchObservedRunningTime="2026-07-30 16:46:29.895629344 +0000 UTC m=+619.153090764"
Jul 30 16:46:29.908672 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.908619    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78fd44b96b-g74vt" podStartSLOduration=1.488019413 podStartE2EDuration="1.908599837s" totalImagesPullingTime="420.580424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:29.907439209 +0000 UTC m=+619.164900629" watchObservedRunningTime="2026-07-30 16:46:29.908599837 +0000 UTC m=+619.166061257"
Jul 30 16:46:29.921747 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:29.921704    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-8lngw" podStartSLOduration=1.486228793 podStartE2EDuration="1.921690175s" totalImagesPullingTime="435.461382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:29.921380887 +0000 UTC m=+619.178842310" watchObservedRunningTime="2026-07-30 16:46:29.921690175 +0000 UTC m=+619.179151597"
Jul 30 16:46:30.141333 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.141305    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:30.149350 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.149323    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:30.273251 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.273223    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlgdb\" (UniqueName: \"kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb\") pod \"b7527a73-5c86-476f-a214-80420923d2dd\" (UID: \"b7527a73-5c86-476f-a214-80420923d2dd\") "
Jul 30 16:46:30.273385 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.273255    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlg2c\" (UniqueName: \"kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c\") pod \"9f8b14e8-0f27-4462-a22c-b547eed14964\" (UID: \"9f8b14e8-0f27-4462-a22c-b547eed14964\") "
Jul 30 16:46:30.275329 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.275300    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb" (OuterVolumeSpecName: "kube-api-access-xlgdb") pod "b7527a73-5c86-476f-a214-80420923d2dd" (UID: "b7527a73-5c86-476f-a214-80420923d2dd"). InnerVolumeSpecName "kube-api-access-xlgdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:30.275478 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.275459    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c" (OuterVolumeSpecName: "kube-api-access-dlg2c") pod "9f8b14e8-0f27-4462-a22c-b547eed14964" (UID: "9f8b14e8-0f27-4462-a22c-b547eed14964"). InnerVolumeSpecName "kube-api-access-dlg2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:30.373827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.373796    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlgdb\" (UniqueName: \"kubernetes.io/projected/b7527a73-5c86-476f-a214-80420923d2dd-kube-api-access-xlgdb\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:46:30.373827 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.373822    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlg2c\" (UniqueName: \"kubernetes.io/projected/9f8b14e8-0f27-4462-a22c-b547eed14964-kube-api-access-dlg2c\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:46:30.888356 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.888321    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f8b14e8-0f27-4462-a22c-b547eed14964" containerID="0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c" exitCode=0
Jul 30 16:46:30.888809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.888367    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78fd44b96b-g74vt"
Jul 30 16:46:30.888809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.888403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78fd44b96b-g74vt" event={"ID":"9f8b14e8-0f27-4462-a22c-b547eed14964","Type":"ContainerDied","Data":"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"}
Jul 30 16:46:30.888809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.888447    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78fd44b96b-g74vt" event={"ID":"9f8b14e8-0f27-4462-a22c-b547eed14964","Type":"ContainerDied","Data":"8053174e9f4b2b28ed61a39d942fc6e1e2bab6d2f71f2651f39590c59e9b5f23"}
Jul 30 16:46:30.888809 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.888465    2561 scope.go:117] "RemoveContainer" containerID="0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"
Jul 30 16:46:30.889647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.889586    2561 generic.go:356] "Generic (PLEG): container finished" podID="b7527a73-5c86-476f-a214-80420923d2dd" containerID="46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa" exitCode=0
Jul 30 16:46:30.889647 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.889631    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8lngw" event={"ID":"b7527a73-5c86-476f-a214-80420923d2dd","Type":"ContainerDied","Data":"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"}
Jul 30 16:46:30.889740 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.889669    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8lngw" event={"ID":"b7527a73-5c86-476f-a214-80420923d2dd","Type":"ContainerDied","Data":"f29ff83a277880495971b8436f57baf6b68b7d109ea721fc0b730ad2440bb37e"}
Jul 30 16:46:30.889740 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.889639    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8lngw"
Jul 30 16:46:30.898594 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.898577    2561 scope.go:117] "RemoveContainer" containerID="0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"
Jul 30 16:46:30.898868 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:46:30.898846    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c\": container with ID starting with 0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c not found: ID does not exist" containerID="0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"
Jul 30 16:46:30.898941 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.898885    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c"} err="failed to get container status \"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c\": rpc error: code = NotFound desc = could not find container \"0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c\": container with ID starting with 0795315bef410f5ebef721c882a92ad65faa210239872f3c8208843bf740da0c not found: ID does not exist"
Jul 30 16:46:30.898941 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.898902    2561 scope.go:117] "RemoveContainer" containerID="46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"
Jul 30 16:46:30.906237 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.906221    2561 scope.go:117] "RemoveContainer" containerID="46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"
Jul 30 16:46:30.906452 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:46:30.906435    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa\": container with ID starting with 46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa not found: ID does not exist" containerID="46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"
Jul 30 16:46:30.906520 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.906456    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa"} err="failed to get container status \"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa\": rpc error: code = NotFound desc = could not find container \"46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa\": container with ID starting with 46ae63ac50a5fc953d9f8cd232658eaa5c5bc88ff347d07a5b038cbe1e698bfa not found: ID does not exist"
Jul 30 16:46:30.910119 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.910098    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:30.914508 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.914477    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78fd44b96b-g74vt"]
Jul 30 16:46:30.924446 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.924428    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:30.926711 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:30.926693    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8lngw"]
Jul 30 16:46:31.340417 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:31.340388    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8b14e8-0f27-4462-a22c-b547eed14964" path="/var/lib/kubelet/pods/9f8b14e8-0f27-4462-a22c-b547eed14964/volumes"
Jul 30 16:46:31.340717 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:46:31.340703    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7527a73-5c86-476f-a214-80420923d2dd" path="/var/lib/kubelet/pods/b7527a73-5c86-476f-a214-80420923d2dd/volumes"
Jul 30 16:47:12.469881 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:47:12.469844    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:47:35.223909 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:47:35.223871    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:47:54.627527 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:47:54.627468    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:48:36.118463 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:36.118429    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:48:44.121831 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.121762    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"]
Jul 30 16:48:44.122288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122183    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8b14e8-0f27-4462-a22c-b547eed14964" containerName="authorino"
Jul 30 16:48:44.122288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122196    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8b14e8-0f27-4462-a22c-b547eed14964" containerName="authorino"
Jul 30 16:48:44.122288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122209    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7527a73-5c86-476f-a214-80420923d2dd" containerName="authorino"
Jul 30 16:48:44.122288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122214    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7527a73-5c86-476f-a214-80420923d2dd" containerName="authorino"
Jul 30 16:48:44.122288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122284    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8b14e8-0f27-4462-a22c-b547eed14964" containerName="authorino"
Jul 30 16:48:44.122447 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.122297    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7527a73-5c86-476f-a214-80420923d2dd" containerName="authorino"
Jul 30 16:48:44.124222 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.124206    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.126689 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.126666    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 16:48:44.127541 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.127522    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 16:48:44.127645 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.127570    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sqbq4\""
Jul 30 16:48:44.127645 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.127621    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 30 16:48:44.134239 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.134218    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"]
Jul 30 16:48:44.231693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231655    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.231849 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231696    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.231849 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231730    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.231849 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231821    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.231977 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231868    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76qw\" (UniqueName: \"kubernetes.io/projected/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-kube-api-access-v76qw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.231977 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.231918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333232 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333200    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333232 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333437 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333437 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333437 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76qw\" (UniqueName: \"kubernetes.io/projected/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-kube-api-access-v76qw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333437 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333324    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333699 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333675    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333738 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333696    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.333772 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.333745    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.335570 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.335552    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.335820 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.335804    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.344542 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.344524    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76qw\" (UniqueName: \"kubernetes.io/projected/ea31c6b4-eedd-44e0-a589-4ba77f822f7e-kube-api-access-v76qw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd\" (UID: \"ea31c6b4-eedd-44e0-a589-4ba77f822f7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.435666 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.435602    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:44.569135 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.569110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"]
Jul 30 16:48:44.571261 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:48:44.571238    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea31c6b4_eedd_44e0_a589_4ba77f822f7e.slice/crio-fc4482646bbd30eef62deffea62cd10c569a1659b4169b6d9b9aa00d19d3255b WatchSource:0}: Error finding container fc4482646bbd30eef62deffea62cd10c569a1659b4169b6d9b9aa00d19d3255b: Status 404 returned error can't find the container with id fc4482646bbd30eef62deffea62cd10c569a1659b4169b6d9b9aa00d19d3255b
Jul 30 16:48:44.572587 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:44.572565    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:48:45.513561 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:45.513526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd" event={"ID":"ea31c6b4-eedd-44e0-a589-4ba77f822f7e","Type":"ContainerStarted","Data":"fc4482646bbd30eef62deffea62cd10c569a1659b4169b6d9b9aa00d19d3255b"}
Jul 30 16:48:46.124924 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:46.124889    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:48:46.519946 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:46.519864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd" event={"ID":"ea31c6b4-eedd-44e0-a589-4ba77f822f7e","Type":"ContainerStarted","Data":"8973fd89452add886412c70f4467dbf97734ac0be3cb4f3357500861165a6a61"}
Jul 30 16:48:46.539944 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:46.539891    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd" podStartSLOduration=1.542467193 podStartE2EDuration="2.539867467s" totalImagesPullingTime="997.400274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:48:46.539282872 +0000 UTC m=+755.796744359" watchObservedRunningTime="2026-07-30 16:48:46.539867467 +0000 UTC m=+755.797328888"
Jul 30 16:48:54.435951 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:54.435913    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:54.436339 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:54.435976    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:54.448793 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:54.448772    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:48:54.567288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:48:54.567261    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd"
Jul 30 16:49:01.723876 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:01.723844    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:49:20.422940 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.422911    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"]
Jul 30 16:49:20.427348 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.427327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.429629 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.429608    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 16:49:20.436574 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.436553    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"]
Jul 30 16:49:20.516406 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.516406 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.516654 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516421    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.516654 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516445    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.516654 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516558    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031f1e58-2320-4655-9610-073f9344bf0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.516654 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.516591    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549mf\" (UniqueName: \"kubernetes.io/projected/031f1e58-2320-4655-9610-073f9344bf0e-kube-api-access-549mf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617362 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617362 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617360    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617631 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617631 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617631 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031f1e58-2320-4655-9610-073f9344bf0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617631 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549mf\" (UniqueName: \"kubernetes.io/projected/031f1e58-2320-4655-9610-073f9344bf0e-kube-api-access-549mf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617834 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617807    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617888 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617845    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.617888 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.617883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.619746 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.619724    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/031f1e58-2320-4655-9610-073f9344bf0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.619909 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.619890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/031f1e58-2320-4655-9610-073f9344bf0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.624681 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.624658    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-549mf\" (UniqueName: \"kubernetes.io/projected/031f1e58-2320-4655-9610-073f9344bf0e-kube-api-access-549mf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j\" (UID: \"031f1e58-2320-4655-9610-073f9344bf0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.736702 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.736621    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:20.870330 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:20.870273    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"]
Jul 30 16:49:20.873142 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:49:20.873115    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod031f1e58_2320_4655_9610_073f9344bf0e.slice/crio-99f96bfe4a96c1c412b7b143f051cfde6e68554650e26842ee03f3b4212846b4 WatchSource:0}: Error finding container 99f96bfe4a96c1c412b7b143f051cfde6e68554650e26842ee03f3b4212846b4: Status 404 returned error can't find the container with id 99f96bfe4a96c1c412b7b143f051cfde6e68554650e26842ee03f3b4212846b4
Jul 30 16:49:21.677622 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:21.677581    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j" event={"ID":"031f1e58-2320-4655-9610-073f9344bf0e","Type":"ContainerStarted","Data":"32fb1e1e38ef8ca1c3aecf7bed4c6fc4b3ee9b5b2fc1bc4a49135d09e396b6d1"}
Jul 30 16:49:21.678090 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:21.677631    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j" event={"ID":"031f1e58-2320-4655-9610-073f9344bf0e","Type":"ContainerStarted","Data":"99f96bfe4a96c1c412b7b143f051cfde6e68554650e26842ee03f3b4212846b4"}
Jul 30 16:49:21.697048 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:21.696989    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j" podStartSLOduration=1.511575304 podStartE2EDuration="1.696969531s" totalImagesPullingTime="185.394227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:49:21.696839496 +0000 UTC m=+790.954300920" watchObservedRunningTime="2026-07-30 16:49:21.696969531 +0000 UTC m=+790.954430953"
Jul 30 16:49:23.120672 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:23.120629    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:49:30.736998 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:30.736963    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:30.737595 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:30.737035    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:30.749049 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:30.749027    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:49:31.734865 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:49:31.734836    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j"
Jul 30 16:50:49.633812 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.633777    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77dbcddb95-8w79v"]
Jul 30 16:50:49.637277 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.637261    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.653043 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.653018    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77dbcddb95-8w79v"]
Jul 30 16:50:49.739780 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.739752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a07c376-a8b6-41b0-891c-6584a1339cd6-tls-cert\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.739909 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.739824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfl5\" (UniqueName: \"kubernetes.io/projected/8a07c376-a8b6-41b0-891c-6584a1339cd6-kube-api-access-khfl5\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.840322 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.840293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a07c376-a8b6-41b0-891c-6584a1339cd6-tls-cert\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.840453 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.840362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khfl5\" (UniqueName: \"kubernetes.io/projected/8a07c376-a8b6-41b0-891c-6584a1339cd6-kube-api-access-khfl5\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.842714 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.842687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a07c376-a8b6-41b0-891c-6584a1339cd6-tls-cert\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.848748 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.848725    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfl5\" (UniqueName: \"kubernetes.io/projected/8a07c376-a8b6-41b0-891c-6584a1339cd6-kube-api-access-khfl5\") pod \"authorino-77dbcddb95-8w79v\" (UID: \"8a07c376-a8b6-41b0-891c-6584a1339cd6\") " pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:49.952694 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:49.952626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77dbcddb95-8w79v"
Jul 30 16:50:50.081719 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:50.081697    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77dbcddb95-8w79v"]
Jul 30 16:50:50.084187 ip-10-0-132-133 kubenswrapper[2561]: W0730 16:50:50.084154    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a07c376_a8b6_41b0_891c_6584a1339cd6.slice/crio-4094a46c7bd1f2d02b0869438d9c72ac3c4a50eb077aa01310cf2518035842e9 WatchSource:0}: Error finding container 4094a46c7bd1f2d02b0869438d9c72ac3c4a50eb077aa01310cf2518035842e9: Status 404 returned error can't find the container with id 4094a46c7bd1f2d02b0869438d9c72ac3c4a50eb077aa01310cf2518035842e9
Jul 30 16:50:51.086263 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.085294    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77dbcddb95-8w79v" event={"ID":"8a07c376-a8b6-41b0-891c-6584a1339cd6","Type":"ContainerStarted","Data":"c0bc06b54669cdfa8ef8a2ab13faa65c82aa9c540a6b189b585b3a170d23cbe5"}
Jul 30 16:50:51.086263 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.085345    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77dbcddb95-8w79v" event={"ID":"8a07c376-a8b6-41b0-891c-6584a1339cd6","Type":"ContainerStarted","Data":"4094a46c7bd1f2d02b0869438d9c72ac3c4a50eb077aa01310cf2518035842e9"}
Jul 30 16:50:51.101794 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.101745    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77dbcddb95-8w79v" podStartSLOduration=1.466648596 podStartE2EDuration="2.101729074s" totalImagesPullingTime="635.080478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:50:51.101134408 +0000 UTC m=+880.358595830" watchObservedRunningTime="2026-07-30 16:50:51.101729074 +0000 UTC m=+880.359190494"
Jul 30 16:50:51.133667 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.132733    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:50:51.133667 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.132955    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bb86cb758-5djx9" podUID="01777f05-1e9e-4327-b002-702fc73384b6" containerName="authorino" containerID="cri-o://81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc" gracePeriod=30
Jul 30 16:50:51.410674 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.410645    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:50:51.558072 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.558043    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert\") pod \"01777f05-1e9e-4327-b002-702fc73384b6\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") "
Jul 30 16:50:51.558256 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.558103    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8cg2\" (UniqueName: \"kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2\") pod \"01777f05-1e9e-4327-b002-702fc73384b6\" (UID: \"01777f05-1e9e-4327-b002-702fc73384b6\") "
Jul 30 16:50:51.560136 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.560099    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2" (OuterVolumeSpecName: "kube-api-access-x8cg2") pod "01777f05-1e9e-4327-b002-702fc73384b6" (UID: "01777f05-1e9e-4327-b002-702fc73384b6"). InnerVolumeSpecName "kube-api-access-x8cg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:50:51.569288 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.569261    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "01777f05-1e9e-4327-b002-702fc73384b6" (UID: "01777f05-1e9e-4327-b002-702fc73384b6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:50:51.659734 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.659654    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01777f05-1e9e-4327-b002-702fc73384b6-tls-cert\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:50:51.659734 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:51.659679    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8cg2\" (UniqueName: \"kubernetes.io/projected/01777f05-1e9e-4327-b002-702fc73384b6-kube-api-access-x8cg2\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 16:50:52.092379 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.092340    2561 generic.go:356] "Generic (PLEG): container finished" podID="01777f05-1e9e-4327-b002-702fc73384b6" containerID="81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc" exitCode=0
Jul 30 16:50:52.092830 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.092392    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb86cb758-5djx9"
Jul 30 16:50:52.092830 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.092422    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb86cb758-5djx9" event={"ID":"01777f05-1e9e-4327-b002-702fc73384b6","Type":"ContainerDied","Data":"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"}
Jul 30 16:50:52.092830 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.092511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb86cb758-5djx9" event={"ID":"01777f05-1e9e-4327-b002-702fc73384b6","Type":"ContainerDied","Data":"86c1a901bc783ed5ce5afdf95625bbb55bcc8f2d431c130905c4f7970db9d554"}
Jul 30 16:50:52.092830 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.092534    2561 scope.go:117] "RemoveContainer" containerID="81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"
Jul 30 16:50:52.103374 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.103356    2561 scope.go:117] "RemoveContainer" containerID="81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"
Jul 30 16:50:52.103709 ip-10-0-132-133 kubenswrapper[2561]: E0730 16:50:52.103689    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc\": container with ID starting with 81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc not found: ID does not exist" containerID="81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"
Jul 30 16:50:52.103805 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.103722    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc"} err="failed to get container status \"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc\": rpc error: code = NotFound desc = could not find container \"81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc\": container with ID starting with 81f243b0ef5588263428bc009460abd60b690da01a4d970e134b7b8d1e90dbcc not found: ID does not exist"
Jul 30 16:50:52.116396 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.116370    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:50:52.120355 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:52.120336    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bb86cb758-5djx9"]
Jul 30 16:50:53.345370 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:50:53.345327    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01777f05-1e9e-4327-b002-702fc73384b6" path="/var/lib/kubelet/pods/01777f05-1e9e-4327-b002-702fc73384b6/volumes"
Jul 30 16:51:11.177970 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:11.177942    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:51:11.179238 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:11.179217    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:51:11.183680 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:11.183660    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:51:11.185127 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:11.185110    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:51:23.232817 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:23.232786    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:51:32.919272 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:32.919238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:51:44.521713 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:51:44.521680    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:53:27.816194 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:53:27.816161    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:54:48.418271 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:54:48.418232    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:54:57.659797 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:54:57.659763    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:55:11.824616 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:55:11.824582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:55:20.919069 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:55:20.919033    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:55:32.342652 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:55:32.342615    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:55:42.118876 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:55:42.118838    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:55:54.717085 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:55:54.717052    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:56:04.321274 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:04.321238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:56:11.203173 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:11.203146    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:56:11.205078 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:11.205057    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 16:56:11.209152 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:11.209133    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:56:11.211077 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:11.211059    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 16:56:29.520342 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:29.520303    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:56:40.214199 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:56:40.214157    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:57:03.019630 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:57:03.019550    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:57:13.921125 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:57:13.921092    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:57:30.312223 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:57:30.312184    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:57:51.313956 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:57:51.313922    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:58:06.117994 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:58:06.117926    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:58:16.123153 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:58:16.123116    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:58:25.717119 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:58:25.717080    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:58:35.315412 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:58:35.315314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:58:44.536693 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:58:44.536653    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:59:05.625404 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:59:05.625368    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:59:16.127232 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:59:16.127200    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 16:59:23.930706 ip-10-0-132-133 kubenswrapper[2561]: I0730 16:59:23.930671    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:00:14.339882 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.339842    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 17:00:14.340366 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.340027    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" podUID="e03fce07-5432-4d60-a584-78f7e53c7768" containerName="manager" containerID="cri-o://23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a" gracePeriod=10
Jul 30 17:00:14.588124 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.588097    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 17:00:14.646009 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.645943    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djtbd\" (UniqueName: \"kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd\") pod \"e03fce07-5432-4d60-a584-78f7e53c7768\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") "
Jul 30 17:00:14.646137 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.646033    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume\") pod \"e03fce07-5432-4d60-a584-78f7e53c7768\" (UID: \"e03fce07-5432-4d60-a584-78f7e53c7768\") "
Jul 30 17:00:14.646439 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.646416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e03fce07-5432-4d60-a584-78f7e53c7768" (UID: "e03fce07-5432-4d60-a584-78f7e53c7768"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:00:14.647934 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.647913    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd" (OuterVolumeSpecName: "kube-api-access-djtbd") pod "e03fce07-5432-4d60-a584-78f7e53c7768" (UID: "e03fce07-5432-4d60-a584-78f7e53c7768"). InnerVolumeSpecName "kube-api-access-djtbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:00:14.680437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.680408    2561 generic.go:356] "Generic (PLEG): container finished" podID="e03fce07-5432-4d60-a584-78f7e53c7768" containerID="23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a" exitCode=0
Jul 30 17:00:14.680553 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.680445    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"
Jul 30 17:00:14.680553 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.680505    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" event={"ID":"e03fce07-5432-4d60-a584-78f7e53c7768","Type":"ContainerDied","Data":"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"}
Jul 30 17:00:14.680625 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.680559    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb" event={"ID":"e03fce07-5432-4d60-a584-78f7e53c7768","Type":"ContainerDied","Data":"389deeeaf391c3f767c716ce2923156a98314801786c0d6a6b7ef3256c3c4155"}
Jul 30 17:00:14.680625 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.680581    2561 scope.go:117] "RemoveContainer" containerID="23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"
Jul 30 17:00:14.690455 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.690439    2561 scope.go:117] "RemoveContainer" containerID="23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"
Jul 30 17:00:14.690747 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:00:14.690730    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a\": container with ID starting with 23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a not found: ID does not exist" containerID="23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"
Jul 30 17:00:14.690805 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.690755    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a"} err="failed to get container status \"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a\": rpc error: code = NotFound desc = could not find container \"23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a\": container with ID starting with 23fe1c70a76df01ea52bf12f6091f930c7a48dfa0932d2820d375728a2a2a43a not found: ID does not exist"
Jul 30 17:00:14.703553 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.703531    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 17:00:14.707853 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.707833    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhlpb"]
Jul 30 17:00:14.746736 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.746716    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03fce07-5432-4d60-a584-78f7e53c7768-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:00:14.746736 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:14.746735    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djtbd\" (UniqueName: \"kubernetes.io/projected/e03fce07-5432-4d60-a584-78f7e53c7768-kube-api-access-djtbd\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:00:15.340420 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:00:15.340392    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03fce07-5432-4d60-a584-78f7e53c7768" path="/var/lib/kubelet/pods/e03fce07-5432-4d60-a584-78f7e53c7768/volumes"
Jul 30 17:01:11.230045 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:11.230019    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:01:11.232430 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:11.232411    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:01:11.238878 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:11.238859    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:01:11.241318 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:11.241300    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:01:15.061917 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.061888    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:01:15.062285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062212    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01777f05-1e9e-4327-b002-702fc73384b6" containerName="authorino"
Jul 30 17:01:15.062285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062224    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="01777f05-1e9e-4327-b002-702fc73384b6" containerName="authorino"
Jul 30 17:01:15.062285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062236    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03fce07-5432-4d60-a584-78f7e53c7768" containerName="manager"
Jul 30 17:01:15.062285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062241    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03fce07-5432-4d60-a584-78f7e53c7768" containerName="manager"
Jul 30 17:01:15.062409 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062294    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03fce07-5432-4d60-a584-78f7e53c7768" containerName="manager"
Jul 30 17:01:15.062409 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.062303    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="01777f05-1e9e-4327-b002-702fc73384b6" containerName="authorino"
Jul 30 17:01:15.065457 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.065437    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.067974 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.067953    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dcm8j\""
Jul 30 17:01:15.084985 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.082703    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:01:15.211214 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.211179    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.211214 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.211216    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vg8\" (UniqueName: \"kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.311970 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.311902    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.311970 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.311937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vg8\" (UniqueName: \"kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.312278 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.312258    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.321063 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.321042    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vg8\" (UniqueName: \"kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8\") pod \"kuadrant-operator-controller-manager-86878c769f-d7xxc\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.375702 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.375674    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.509885 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.509856    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:01:15.513677 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:01:15.513649    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47281a85_af87_4dc7_bed1_c36c7f1bdd55.slice/crio-24d2ae4d7206f95d355282be8c51a9e2fe3c2c3c7d9c5eb37406d3a0827fff82 WatchSource:0}: Error finding container 24d2ae4d7206f95d355282be8c51a9e2fe3c2c3c7d9c5eb37406d3a0827fff82: Status 404 returned error can't find the container with id 24d2ae4d7206f95d355282be8c51a9e2fe3c2c3c7d9c5eb37406d3a0827fff82
Jul 30 17:01:15.951907 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.951871    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" event={"ID":"47281a85-af87-4dc7-bed1-c36c7f1bdd55","Type":"ContainerStarted","Data":"2306e5c48bb269e62172f8ba95e2c97e7bfb0b76a56b37080baeb55b0fb6a642"}
Jul 30 17:01:15.952085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.951922    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" event={"ID":"47281a85-af87-4dc7-bed1-c36c7f1bdd55","Type":"ContainerStarted","Data":"24d2ae4d7206f95d355282be8c51a9e2fe3c2c3c7d9c5eb37406d3a0827fff82"}
Jul 30 17:01:15.952085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.951988    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:15.970929 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:15.970881    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" podStartSLOduration=0.970864563 podStartE2EDuration="970.864563ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:01:15.970283714 +0000 UTC m=+1505.227745155" watchObservedRunningTime="2026-07-30 17:01:15.970864563 +0000 UTC m=+1505.228325984"
Jul 30 17:01:26.959439 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:26.959401    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:01:37.110933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:37.110861    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:01:45.725884 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:01:45.725850    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:02:08.455326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:02:08.455289    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:02:46.729708 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:02:46.729674    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:02:57.434107 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:02:57.434075    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:03:08.122858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:03:08.122823    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:03:17.922648 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:03:17.922614    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:03:29.818366 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:03:29.818332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:03:37.325832 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:03:37.325800    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:03:48.623687 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:03:48.623653    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:00.220674 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:00.220635    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:09.923847 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:09.923808    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:26.016583 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:26.016530    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:35.123657 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:35.123570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:38.428824 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.428789    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:04:38.432683 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.432665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.435647 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.435623    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-81d0bf9d-kserve-self-signed-certs\""
Jul 30 17:04:38.450141 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.450117    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:04:38.484827 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.484796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.484962 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.484831    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtp9s\" (UniqueName: \"kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.484962 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.484937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.485053 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.484964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.485053 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.484998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.485053 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.485027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585495 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585506    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585530    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585609    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtp9s\" (UniqueName: \"kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585980 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585942    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.585980 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.585967    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.586180 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.586024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.587683 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.587664    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.587959 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.587936    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.593774 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.593755    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtp9s\" (UniqueName: \"kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s\") pod \"e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") " pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.742359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.742287    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:38.875569 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.875535    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:04:38.879500 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:04:38.879459    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod123b972e_f807_4818_addd_9384258deda5.slice/crio-5f0eada2b501e9f479c5f36815ccaadc09e6a0142f627e7c80e66d7eb9ca876a WatchSource:0}: Error finding container 5f0eada2b501e9f479c5f36815ccaadc09e6a0142f627e7c80e66d7eb9ca876a: Status 404 returned error can't find the container with id 5f0eada2b501e9f479c5f36815ccaadc09e6a0142f627e7c80e66d7eb9ca876a
Jul 30 17:04:38.890753 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:38.890728    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" event={"ID":"123b972e-f807-4818-addd-9384258deda5","Type":"ContainerStarted","Data":"5f0eada2b501e9f479c5f36815ccaadc09e6a0142f627e7c80e66d7eb9ca876a"}
Jul 30 17:04:39.896709 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:39.896676    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" event={"ID":"123b972e-f807-4818-addd-9384258deda5","Type":"ContainerStarted","Data":"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"}
Jul 30 17:04:39.916613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:39.916561    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" podStartSLOduration=1.916543492 podStartE2EDuration="1.916543492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:04:39.914748314 +0000 UTC m=+1709.172209738" watchObservedRunningTime="2026-07-30 17:04:39.916543492 +0000 UTC m=+1709.174004912"
Jul 30 17:04:47.163070 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.163031    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:04:47.166930 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.166909    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.169395 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.169371    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-81d0bf9d-kserve-self-signed-certs\""
Jul 30 17:04:47.176183 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.176164    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:04:47.257746 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257721    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.257863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257747    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.257863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257766    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.257863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257792    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7mkr\" (UniqueName: \"kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.257863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257841    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.258004 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.257866    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359095 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359069    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359130    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359148    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359187    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7mkr\" (UniqueName: \"kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359580 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359516    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359640 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.359731 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.359698    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.361704 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.361652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.362148 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.362126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.367885 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.367856    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7mkr\" (UniqueName: \"kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr\") pod \"e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") " pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.477871 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.477810    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:47.615301 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.615273    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:04:47.616965 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:04:47.616937    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb84d7cad_c0ea_4e82_af28_767c449dd5bf.slice/crio-80f56f93cc5eb89d69251a00a83c22ee2e8ce132bd11a9d57ee543af31a6f36d WatchSource:0}: Error finding container 80f56f93cc5eb89d69251a00a83c22ee2e8ce132bd11a9d57ee543af31a6f36d: Status 404 returned error can't find the container with id 80f56f93cc5eb89d69251a00a83c22ee2e8ce132bd11a9d57ee543af31a6f36d
Jul 30 17:04:47.937252 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.937219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" event={"ID":"b84d7cad-c0ea-4e82-af28-767c449dd5bf","Type":"ContainerStarted","Data":"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"}
Jul 30 17:04:47.937417 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.937265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" event={"ID":"b84d7cad-c0ea-4e82-af28-767c449dd5bf","Type":"ContainerStarted","Data":"80f56f93cc5eb89d69251a00a83c22ee2e8ce132bd11a9d57ee543af31a6f36d"}
Jul 30 17:04:47.955078 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:47.955031    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" podStartSLOduration=0.955015193 podStartE2EDuration="955.015193ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:04:47.954234119 +0000 UTC m=+1717.211695540" watchObservedRunningTime="2026-07-30 17:04:47.955015193 +0000 UTC m=+1717.212476613"
Jul 30 17:04:48.619078 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:48.619044    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:04:48.742789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:48.742724    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:48.742789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:48.742765    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:48.755128 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:48.755105    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:48.953447 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:48.953405    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:04:57.478455 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:57.478413    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:57.478952 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:57.478515    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:57.490890 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:57.490866    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:04:57.999160 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:04:57.999130    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:05:00.523917 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:00.523884    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:05:10.393969 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:10.393936    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:05:11.489618 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:11.489587    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:05:14.652277 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:14.652237    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:05:25.413577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.413539    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" podUID="123b972e-f807-4818-addd-9384258deda5" containerName="main" containerID="cri-o://b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab" gracePeriod=45
Jul 30 17:05:25.668934 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.668870    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:05:25.683629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683607    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683644    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683669    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtp9s\" (UniqueName: \"kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683685    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683839    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683883    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs\") pod \"123b972e-f807-4818-addd-9384258deda5\" (UID: \"123b972e-f807-4818-addd-9384258deda5\") "
Jul 30 17:05:25.683938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683886    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:25.683938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.683902    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home" (OuterVolumeSpecName: "home") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:25.684133 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.684058    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache" (OuterVolumeSpecName: "model-cache") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:25.684133 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.684113    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:25.684133 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.684129    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:25.686554 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.685869    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:05:25.686920 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.686261    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s" (OuterVolumeSpecName: "kube-api-access-mtp9s") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "kube-api-access-mtp9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:05:25.686920 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.686773    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm" (OuterVolumeSpecName: "dshm") pod "123b972e-f807-4818-addd-9384258deda5" (UID: "123b972e-f807-4818-addd-9384258deda5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:25.785390 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.785356    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:25.785390 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.785384    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123b972e-f807-4818-addd-9384258deda5-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:25.785390 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.785394    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtp9s\" (UniqueName: \"kubernetes.io/projected/123b972e-f807-4818-addd-9384258deda5-kube-api-access-mtp9s\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:25.785649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:25.785403    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123b972e-f807-4818-addd-9384258deda5-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.124471 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.124439    2561 generic.go:356] "Generic (PLEG): container finished" podID="123b972e-f807-4818-addd-9384258deda5" containerID="b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab" exitCode=0
Jul 30 17:05:26.124635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.124479    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"
Jul 30 17:05:26.124635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.124530    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" event={"ID":"123b972e-f807-4818-addd-9384258deda5","Type":"ContainerDied","Data":"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"}
Jul 30 17:05:26.124635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.124572    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw" event={"ID":"123b972e-f807-4818-addd-9384258deda5","Type":"ContainerDied","Data":"5f0eada2b501e9f479c5f36815ccaadc09e6a0142f627e7c80e66d7eb9ca876a"}
Jul 30 17:05:26.124635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.124590    2561 scope.go:117] "RemoveContainer" containerID="b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"
Jul 30 17:05:26.133512 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.133477    2561 scope.go:117] "RemoveContainer" containerID="b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"
Jul 30 17:05:26.133792 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:05:26.133775    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab\": container with ID starting with b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab not found: ID does not exist" containerID="b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"
Jul 30 17:05:26.133845 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.133799    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab"} err="failed to get container status \"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab\": rpc error: code = NotFound desc = could not find container \"b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab\": container with ID starting with b946490b77c40993ef5f9f4a6f629bda57cbc326d5b48300e5321ea733bc90ab not found: ID does not exist"
Jul 30 17:05:26.145143 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.145121    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:05:26.149218 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.149201    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-81d0bf9d-kserve-7555956bc9-kcckw"]
Jul 30 17:05:26.508477 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.508381    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" podUID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" containerName="main" containerID="cri-o://ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d" gracePeriod=45
Jul 30 17:05:26.760319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.760259    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:05:26.791389 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791361    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791395    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791427    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791458    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7mkr\" (UniqueName: \"kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791501    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791745 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791536    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir\") pod \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\" (UID: \"b84d7cad-c0ea-4e82-af28-767c449dd5bf\") "
Jul 30 17:05:26.791745 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791647    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home" (OuterVolumeSpecName: "home") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:26.791745 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791677    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:26.791899 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791779    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.791899 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791832    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.791899 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.791866    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:26.793659 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.793631    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm" (OuterVolumeSpecName: "dshm") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:05:26.793659 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.793638    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr" (OuterVolumeSpecName: "kube-api-access-k7mkr") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "kube-api-access-k7mkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:05:26.793795 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.793704    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b84d7cad-c0ea-4e82-af28-767c449dd5bf" (UID: "b84d7cad-c0ea-4e82-af28-767c449dd5bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:05:26.892440 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.892404    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.892440 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.892433    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.892440 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.892444    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b84d7cad-c0ea-4e82-af28-767c449dd5bf-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:26.892671 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:26.892456    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7mkr\" (UniqueName: \"kubernetes.io/projected/b84d7cad-c0ea-4e82-af28-767c449dd5bf-kube-api-access-k7mkr\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:05:27.131369 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.131333    2561 generic.go:356] "Generic (PLEG): container finished" podID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" containerID="ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d" exitCode=0
Jul 30 17:05:27.131577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.131382    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"
Jul 30 17:05:27.131577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.131397    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" event={"ID":"b84d7cad-c0ea-4e82-af28-767c449dd5bf","Type":"ContainerDied","Data":"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"}
Jul 30 17:05:27.131577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.131429    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65" event={"ID":"b84d7cad-c0ea-4e82-af28-767c449dd5bf","Type":"ContainerDied","Data":"80f56f93cc5eb89d69251a00a83c22ee2e8ce132bd11a9d57ee543af31a6f36d"}
Jul 30 17:05:27.131577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.131446    2561 scope.go:117] "RemoveContainer" containerID="ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"
Jul 30 17:05:27.140622 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.140601    2561 scope.go:117] "RemoveContainer" containerID="ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"
Jul 30 17:05:27.140839 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:05:27.140821    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d\": container with ID starting with ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d not found: ID does not exist" containerID="ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"
Jul 30 17:05:27.140884 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.140850    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d"} err="failed to get container status \"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d\": rpc error: code = NotFound desc = could not find container \"ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d\": container with ID starting with ddd297f4865a8fcdfa450bf1c0afa90b97073e9bc4ed3750a84f893557e5d60d not found: ID does not exist"
Jul 30 17:05:27.153224 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.153200    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:05:27.156855 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.156829    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-81d0bf9d-kserve-564fb8f755-t5s65"]
Jul 30 17:05:27.340843 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.340809    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123b972e-f807-4818-addd-9384258deda5" path="/var/lib/kubelet/pods/123b972e-f807-4818-addd-9384258deda5/volumes"
Jul 30 17:05:27.341199 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:27.341186    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" path="/var/lib/kubelet/pods/b84d7cad-c0ea-4e82-af28-767c449dd5bf/volumes"
Jul 30 17:05:28.031277 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:28.031235    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:05:34.526999 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:34.526962    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:05:45.837805 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:05:45.837772    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:01.224959 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:01.224928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:11.260767 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:11.260699    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:06:11.265005 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:11.264974    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:06:11.276266 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:11.276211    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:06:11.279714 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:11.279689    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:06:17.228169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:17.228133    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:28.420319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:28.420284    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:38.430599 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:38.430564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:47.451123 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:47.451089    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:06:56.926894 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:06:56.926849    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:07:10.433551 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:07:10.433514    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:07:19.974218 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:07:19.974175    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:07:29.614636 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:07:29.614601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:07:39.226074 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:07:39.225981    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:07:52.934796 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:07:52.934756    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:08:01.941714 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:08:01.941672    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:08:23.922877 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:08:23.922842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:08:36.325644 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:08:36.325608    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:08:51.341450 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:08:51.341418    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:09:17.027910 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:09:17.027868    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:09:26.418165 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:09:26.418131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:09:38.324767 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:09:38.324737    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:09:54.124832 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:09:54.124797    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:10:03.717314 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:03.717281    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:10:14.016693 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:14.016656    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:10:24.823433 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:24.823390    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:10:42.619907 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.619818    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"]
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620297    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123b972e-f807-4818-addd-9384258deda5" containerName="main"
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620315    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="123b972e-f807-4818-addd-9384258deda5" containerName="main"
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620345    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" containerName="main"
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620355    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" containerName="main"
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620444    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="123b972e-f807-4818-addd-9384258deda5" containerName="main"
Jul 30 17:10:42.620479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.620457    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b84d7cad-c0ea-4e82-af28-767c449dd5bf" containerName="main"
Jul 30 17:10:42.623562 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.623544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:42.625956 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.625933    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:10:42.626047 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.625953    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:10:42.631746 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.631721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"]
Jul 30 17:10:42.772112 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.772070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7\" (UID: \"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:42.873131 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.873033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7\" (UID: \"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:42.873749 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.873727    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7\" (UID: \"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:42.934476 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:42.934442    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:43.275065 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:43.275037    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"]
Jul 30 17:10:43.276997 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:10:43.276965    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dfa8fd_4d20_4bfc_b6c5_25d4cfe820c3.slice/crio-aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3 WatchSource:0}: Error finding container aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3: Status 404 returned error can't find the container with id aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3
Jul 30 17:10:43.278958 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:43.278937    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:10:43.611376 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:43.611330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7" event={"ID":"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3","Type":"ContainerStarted","Data":"aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3"}
Jul 30 17:10:46.629619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:46.629585    2561 generic.go:356] "Generic (PLEG): container finished" podID="e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" containerID="9f1433a93770197e2d05db78029f095a5ba214fd068fd039bd8af43205e48485" exitCode=0
Jul 30 17:10:46.630015 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:46.629668    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7" event={"ID":"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3","Type":"ContainerDied","Data":"9f1433a93770197e2d05db78029f095a5ba214fd068fd039bd8af43205e48485"}
Jul 30 17:10:47.766969 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:47.766946    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:47.914599 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:47.914506    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca\") pod \"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3\" (UID: \"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3\") "
Jul 30 17:10:47.914892 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:47.914868    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" (UID: "e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:10:48.016026 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:48.015982    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:10:48.641151 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:48.641120    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"
Jul 30 17:10:48.641151 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:48.641129    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7" event={"ID":"e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3","Type":"ContainerDied","Data":"aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3"}
Jul 30 17:10:48.641361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:48.641179    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa7ba26b70e0e737f9a6970329dcbf02b5d8931c5a120f81174baa2c3cff70d3"
Jul 30 17:10:58.848898 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:58.848861    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"]
Jul 30 17:10:58.852576 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:58.852550    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-36c7a10f-9vfp7"]
Jul 30 17:10:59.340785 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:10:59.340754    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" path="/var/lib/kubelet/pods/e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3/volumes"
Jul 30 17:11:02.430524 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:02.430477    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:11:11.308263 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:11.308235    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:11:11.309233 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:11.309212    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:11:11.314152 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:11.314134    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:11:11.314931 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:11.314914    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:11:12.626940 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:12.626908    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:11:27.486383 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.486344    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"]
Jul 30 17:11:27.486821 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.486689    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" containerName="revoke-keys"
Jul 30 17:11:27.486821 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.486700    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" containerName="revoke-keys"
Jul 30 17:11:27.486821 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.486769    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dfa8fd-4d20-4bfc-b6c5-25d4cfe820c3" containerName="revoke-keys"
Jul 30 17:11:27.494394 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.494376    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:27.497277 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.497254    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:11:27.497402 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.497254    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:11:27.497529 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.497508    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"]
Jul 30 17:11:27.554198 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.554168    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv\" (UID: \"57565732-3459-4ff4-afd8-c8b4d5337b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:27.654743 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.654711    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv\" (UID: \"57565732-3459-4ff4-afd8-c8b4d5337b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:27.655282 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.655263    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv\" (UID: \"57565732-3459-4ff4-afd8-c8b4d5337b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:27.807378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.807290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:27.938515 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:27.938352    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"]
Jul 30 17:11:27.941220 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:11:27.941190    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57565732_3459_4ff4_afd8_c8b4d5337b41.slice/crio-20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6 WatchSource:0}: Error finding container 20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6: Status 404 returned error can't find the container with id 20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6
Jul 30 17:11:28.821944 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:28.821906    2561 generic.go:356] "Generic (PLEG): container finished" podID="57565732-3459-4ff4-afd8-c8b4d5337b41" containerID="7a2b9d2e41d0355fda6e0ec95357bce193b1c33c2544f7af7155ec1c4cf19399" exitCode=0
Jul 30 17:11:28.822361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:28.821976    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv" event={"ID":"57565732-3459-4ff4-afd8-c8b4d5337b41","Type":"ContainerDied","Data":"7a2b9d2e41d0355fda6e0ec95357bce193b1c33c2544f7af7155ec1c4cf19399"}
Jul 30 17:11:28.822361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:28.822061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv" event={"ID":"57565732-3459-4ff4-afd8-c8b4d5337b41","Type":"ContainerStarted","Data":"20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6"}
Jul 30 17:11:29.964174 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:29.964151    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:30.076069 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.076033    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca\") pod \"57565732-3459-4ff4-afd8-c8b4d5337b41\" (UID: \"57565732-3459-4ff4-afd8-c8b4d5337b41\") "
Jul 30 17:11:30.076463 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.076441    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "57565732-3459-4ff4-afd8-c8b4d5337b41" (UID: "57565732-3459-4ff4-afd8-c8b4d5337b41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:11:30.177132 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.177097    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57565732-3459-4ff4-afd8-c8b4d5337b41-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:11:30.832299 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.832274    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"
Jul 30 17:11:30.832475 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.832316    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv" event={"ID":"57565732-3459-4ff4-afd8-c8b4d5337b41","Type":"ContainerDied","Data":"20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6"}
Jul 30 17:11:30.832475 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:30.832361    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20303432f92c83eb383d5376f4ddb782aec00c243414ba08fc01fab55e0a10c6"
Jul 30 17:11:34.293552 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:34.293512    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"]
Jul 30 17:11:34.296068 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:34.296044    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b68b3510-zrfxv"]
Jul 30 17:11:35.341833 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:35.341799    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57565732-3459-4ff4-afd8-c8b4d5337b41" path="/var/lib/kubelet/pods/57565732-3459-4ff4-afd8-c8b4d5337b41/volumes"
Jul 30 17:11:38.129085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:38.129044    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:11:47.227501 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:11:47.227460    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:12:03.404608 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.404527    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"]
Jul 30 17:12:03.405063 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.404862    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57565732-3459-4ff4-afd8-c8b4d5337b41" containerName="revoke-keys"
Jul 30 17:12:03.405063 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.404874    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="57565732-3459-4ff4-afd8-c8b4d5337b41" containerName="revoke-keys"
Jul 30 17:12:03.405063 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.404932    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="57565732-3459-4ff4-afd8-c8b4d5337b41" containerName="revoke-keys"
Jul 30 17:12:03.408080 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.408064    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:03.410645 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.410621    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:12:03.410768 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.410654    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:12:03.425819 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.425796    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"]
Jul 30 17:12:03.439397 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.439371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq\" (UID: \"944055e3-d4f3-4347-8a5d-a3e082904cf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:03.540136 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.540106    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq\" (UID: \"944055e3-d4f3-4347-8a5d-a3e082904cf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:03.540726 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.540706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq\" (UID: \"944055e3-d4f3-4347-8a5d-a3e082904cf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:03.718450 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.718362    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:03.864933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.864805    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"]
Jul 30 17:12:03.868142 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:12:03.868111    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944055e3_d4f3_4347_8a5d_a3e082904cf7.slice/crio-26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9 WatchSource:0}: Error finding container 26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9: Status 404 returned error can't find the container with id 26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9
Jul 30 17:12:03.988361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:03.988279    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq" event={"ID":"944055e3-d4f3-4347-8a5d-a3e082904cf7","Type":"ContainerStarted","Data":"26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9"}
Jul 30 17:12:04.994837 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:04.994800    2561 generic.go:356] "Generic (PLEG): container finished" podID="944055e3-d4f3-4347-8a5d-a3e082904cf7" containerID="bc789928a58285ee39494acf0732e3475cbd563729e6e28ec1f6ba543c3c44a5" exitCode=0
Jul 30 17:12:04.995238 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:04.994884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq" event={"ID":"944055e3-d4f3-4347-8a5d-a3e082904cf7","Type":"ContainerDied","Data":"bc789928a58285ee39494acf0732e3475cbd563729e6e28ec1f6ba543c3c44a5"}
Jul 30 17:12:06.138000 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:06.137975    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:06.157905 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:06.157880    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca\") pod \"944055e3-d4f3-4347-8a5d-a3e082904cf7\" (UID: \"944055e3-d4f3-4347-8a5d-a3e082904cf7\") "
Jul 30 17:12:06.158291 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:06.158259    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "944055e3-d4f3-4347-8a5d-a3e082904cf7" (UID: "944055e3-d4f3-4347-8a5d-a3e082904cf7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:06.258974 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:06.258894    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/944055e3-d4f3-4347-8a5d-a3e082904cf7-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:12:07.005995 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:07.005964    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"
Jul 30 17:12:07.005995 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:07.005965    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq" event={"ID":"944055e3-d4f3-4347-8a5d-a3e082904cf7","Type":"ContainerDied","Data":"26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9"}
Jul 30 17:12:07.006201 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:07.006051    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26a9ad5e8038cdc2b6c653fda788bbc14562655e49ab545a9bdd39e72c3a37e9"
Jul 30 17:12:09.684557 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:09.684524    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"]
Jul 30 17:12:09.689643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:09.689618    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2e888e9-2jdlq"]
Jul 30 17:12:11.340548 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:11.340518    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="944055e3-d4f3-4347-8a5d-a3e082904cf7" path="/var/lib/kubelet/pods/944055e3-d4f3-4347-8a5d-a3e082904cf7/volumes"
Jul 30 17:12:13.623982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:13.623949    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:12:25.021700 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:25.021662    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:12:38.649186 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.649151    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"]
Jul 30 17:12:38.649794 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.649655    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944055e3-d4f3-4347-8a5d-a3e082904cf7" containerName="revoke-keys"
Jul 30 17:12:38.649794 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.649675    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="944055e3-d4f3-4347-8a5d-a3e082904cf7" containerName="revoke-keys"
Jul 30 17:12:38.649794 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.649784    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="944055e3-d4f3-4347-8a5d-a3e082904cf7" containerName="revoke-keys"
Jul 30 17:12:38.652716 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.652696    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:38.654906 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.654886    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:12:38.655003 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.654979    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:12:38.659349 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.659328    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"]
Jul 30 17:12:38.729601 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.729575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4548eef0-xb66m\" (UID: \"dc743bbd-3ca6-46fe-9047-33c512322e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:38.830915 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.830880    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4548eef0-xb66m\" (UID: \"dc743bbd-3ca6-46fe-9047-33c512322e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:38.831472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.831453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4548eef0-xb66m\" (UID: \"dc743bbd-3ca6-46fe-9047-33c512322e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:38.964217 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:38.964130    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:39.098797 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:39.098725    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"]
Jul 30 17:12:39.101720 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:12:39.101689    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc743bbd_3ca6_46fe_9047_33c512322e7b.slice/crio-8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4 WatchSource:0}: Error finding container 8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4: Status 404 returned error can't find the container with id 8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4
Jul 30 17:12:39.154829 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:39.154800    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m" event={"ID":"dc743bbd-3ca6-46fe-9047-33c512322e7b","Type":"ContainerStarted","Data":"8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4"}
Jul 30 17:12:40.161050 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:40.161015    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc743bbd-3ca6-46fe-9047-33c512322e7b" containerID="8bfb4998eee9bd2b57c46f59d49f3af5c7968530dbba310dc098af2471261c54" exitCode=0
Jul 30 17:12:40.161445 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:40.161084    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m" event={"ID":"dc743bbd-3ca6-46fe-9047-33c512322e7b","Type":"ContainerDied","Data":"8bfb4998eee9bd2b57c46f59d49f3af5c7968530dbba310dc098af2471261c54"}
Jul 30 17:12:41.298085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:41.298056    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:41.453428 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:41.453336    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca\") pod \"dc743bbd-3ca6-46fe-9047-33c512322e7b\" (UID: \"dc743bbd-3ca6-46fe-9047-33c512322e7b\") "
Jul 30 17:12:41.453695 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:41.453673    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc743bbd-3ca6-46fe-9047-33c512322e7b" (UID: "dc743bbd-3ca6-46fe-9047-33c512322e7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:41.554163 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:41.554123    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc743bbd-3ca6-46fe-9047-33c512322e7b-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:12:42.171228 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:42.171195    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"
Jul 30 17:12:42.171391 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:42.171228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m" event={"ID":"dc743bbd-3ca6-46fe-9047-33c512322e7b","Type":"ContainerDied","Data":"8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4"}
Jul 30 17:12:42.171391 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:42.171269    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8835435c7f7bd0ed14bd382bf2c232a2e10d5714d0154aa98c706e82b1a22de4"
Jul 30 17:12:46.033027 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:46.032993    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"]
Jul 30 17:12:46.050851 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:46.050827    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4548eef0-xb66m"]
Jul 30 17:12:47.340747 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:47.340717    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc743bbd-3ca6-46fe-9047-33c512322e7b" path="/var/lib/kubelet/pods/dc743bbd-3ca6-46fe-9047-33c512322e7b/volumes"
Jul 30 17:12:49.323576 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:12:49.323539    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:13:11.926395 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:13:11.926359    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:14:42.920068 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:14:42.920030    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:14:53.426251 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:14:53.426218    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:15:00.132983 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.132949    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"]
Jul 30 17:15:00.133367 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.133307    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc743bbd-3ca6-46fe-9047-33c512322e7b" containerName="revoke-keys"
Jul 30 17:15:00.133367 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.133318    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc743bbd-3ca6-46fe-9047-33c512322e7b" containerName="revoke-keys"
Jul 30 17:15:00.133441 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.133370    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc743bbd-3ca6-46fe-9047-33c512322e7b" containerName="revoke-keys"
Jul 30 17:15:00.136699 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.136683    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"
Jul 30 17:15:00.139121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.139091    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:15:00.144560 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.144538    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"]
Jul 30 17:15:00.147277 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.147262    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"
Jul 30 17:15:00.287273 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.287249    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"]
Jul 30 17:15:00.289647 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:15:00.289607    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae1a241_397a_4bf1_8558_1b6b5d524fb1.slice/crio-b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608 WatchSource:0}: Error finding container b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608: Status 404 returned error can't find the container with id b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608
Jul 30 17:15:00.844670 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.844635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm" event={"ID":"5ae1a241-397a-4bf1-8558-1b6b5d524fb1","Type":"ContainerStarted","Data":"149edc7f21bf4bd5cb69ed2d3843b59d77ce31d1c792de2cd8efa2d40c60e4b8"}
Jul 30 17:15:00.844865 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.844692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm" event={"ID":"5ae1a241-397a-4bf1-8558-1b6b5d524fb1","Type":"ContainerStarted","Data":"b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608"}
Jul 30 17:15:00.866157 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:00.866110    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm" podStartSLOduration=0.866095302 podStartE2EDuration="866.095302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:15:00.865654763 +0000 UTC m=+2330.123116184" watchObservedRunningTime="2026-07-30 17:15:00.866095302 +0000 UTC m=+2330.123556722"
Jul 30 17:15:01.850661 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:01.850620    2561 generic.go:356] "Generic (PLEG): container finished" podID="5ae1a241-397a-4bf1-8558-1b6b5d524fb1" containerID="149edc7f21bf4bd5cb69ed2d3843b59d77ce31d1c792de2cd8efa2d40c60e4b8" exitCode=0
Jul 30 17:15:01.851061 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:01.850693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm" event={"ID":"5ae1a241-397a-4bf1-8558-1b6b5d524fb1","Type":"ContainerDied","Data":"149edc7f21bf4bd5cb69ed2d3843b59d77ce31d1c792de2cd8efa2d40c60e4b8"}
Jul 30 17:15:02.993426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:02.993401    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"
Jul 30 17:15:03.861519 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:03.861467    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm"
Jul 30 17:15:03.861519 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:03.861506    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757195-kldvm" event={"ID":"5ae1a241-397a-4bf1-8558-1b6b5d524fb1","Type":"ContainerDied","Data":"b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608"}
Jul 30 17:15:03.861771 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:03.861557    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1ce80808d53e2dfaeafbeac07860b07e84af75a348b22a7b0a4d89a7b63e608"
Jul 30 17:15:06.427520 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:06.427474    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:15:31.749533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.749481    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"]
Jul 30 17:15:31.750047 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.749974    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae1a241-397a-4bf1-8558-1b6b5d524fb1" containerName="cleanup"
Jul 30 17:15:31.750047 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.749991    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae1a241-397a-4bf1-8558-1b6b5d524fb1" containerName="cleanup"
Jul 30 17:15:31.750135 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.750065    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae1a241-397a-4bf1-8558-1b6b5d524fb1" containerName="cleanup"
Jul 30 17:15:31.753077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.753054    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:31.755850 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.755831    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:15:31.756169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.756149    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:15:31.768918 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.768894    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"]
Jul 30 17:15:31.866131 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.866102    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq\" (UID: \"dc0f022d-8a34-4997-9b68-046059a14a71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:31.967398 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.967368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq\" (UID: \"dc0f022d-8a34-4997-9b68-046059a14a71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:31.968042 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:31.968021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq\" (UID: \"dc0f022d-8a34-4997-9b68-046059a14a71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:32.064638 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:32.064572    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:32.195234 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:32.195206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"]
Jul 30 17:15:32.197137 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:15:32.197108    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0f022d_8a34_4997_9b68_046059a14a71.slice/crio-37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc WatchSource:0}: Error finding container 37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc: Status 404 returned error can't find the container with id 37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc
Jul 30 17:15:33.004090 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:33.004053    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc0f022d-8a34-4997-9b68-046059a14a71" containerID="f70f3ecf094b987cdf58c53da2ffaa5ec70fac50906c1d216da99455e8e62dbe" exitCode=0
Jul 30 17:15:33.004474 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:33.004099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq" event={"ID":"dc0f022d-8a34-4997-9b68-046059a14a71","Type":"ContainerDied","Data":"f70f3ecf094b987cdf58c53da2ffaa5ec70fac50906c1d216da99455e8e62dbe"}
Jul 30 17:15:33.004474 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:33.004140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq" event={"ID":"dc0f022d-8a34-4997-9b68-046059a14a71","Type":"ContainerStarted","Data":"37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc"}
Jul 30 17:15:34.136276 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:34.136248    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:34.285349 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:34.285278    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca\") pod \"dc0f022d-8a34-4997-9b68-046059a14a71\" (UID: \"dc0f022d-8a34-4997-9b68-046059a14a71\") "
Jul 30 17:15:34.285638 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:34.285617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc0f022d-8a34-4997-9b68-046059a14a71" (UID: "dc0f022d-8a34-4997-9b68-046059a14a71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:15:34.386254 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:34.386227    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0f022d-8a34-4997-9b68-046059a14a71-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:15:35.015415 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:35.015387    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"
Jul 30 17:15:35.015582 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:35.015425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq" event={"ID":"dc0f022d-8a34-4997-9b68-046059a14a71","Type":"ContainerDied","Data":"37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc"}
Jul 30 17:15:35.015582 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:35.015478    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37ce66091116d9250680a00ef3e9a55121f6d9de3bb0d85768df7068aa6debfc"
Jul 30 17:15:42.494524 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:42.494473    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"]
Jul 30 17:15:42.497503 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:42.497460    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e6f2963a-6ncxq"]
Jul 30 17:15:43.340199 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:43.340165    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc0f022d-8a34-4997-9b68-046059a14a71" path="/var/lib/kubelet/pods/dc0f022d-8a34-4997-9b68-046059a14a71/volumes"
Jul 30 17:15:45.423252 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:15:45.423216    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:16:11.335814 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:11.335781    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:16:11.339789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:11.339762    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:16:11.342550 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:11.342528    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:16:11.346132 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:11.346109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:16:21.725464 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:21.725426    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:16:30.427437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:30.427404    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:16:30.427922 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:30.427659    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" podUID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" containerName="manager" containerID="cri-o://2306e5c48bb269e62172f8ba95e2c97e7bfb0b76a56b37080baeb55b0fb6a642" gracePeriod=10
Jul 30 17:16:31.282300 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.282268    2561 generic.go:356] "Generic (PLEG): container finished" podID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" containerID="2306e5c48bb269e62172f8ba95e2c97e7bfb0b76a56b37080baeb55b0fb6a642" exitCode=0
Jul 30 17:16:31.282439 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.282327    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" event={"ID":"47281a85-af87-4dc7-bed1-c36c7f1bdd55","Type":"ContainerDied","Data":"2306e5c48bb269e62172f8ba95e2c97e7bfb0b76a56b37080baeb55b0fb6a642"}
Jul 30 17:16:31.784914 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.784892    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:16:31.879532 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.879499    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72vg8\" (UniqueName: \"kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8\") pod \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") "
Jul 30 17:16:31.879702 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.879579    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume\") pod \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\" (UID: \"47281a85-af87-4dc7-bed1-c36c7f1bdd55\") "
Jul 30 17:16:31.879963 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.879932    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "47281a85-af87-4dc7-bed1-c36c7f1bdd55" (UID: "47281a85-af87-4dc7-bed1-c36c7f1bdd55"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:16:31.881421 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.881398    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8" (OuterVolumeSpecName: "kube-api-access-72vg8") pod "47281a85-af87-4dc7-bed1-c36c7f1bdd55" (UID: "47281a85-af87-4dc7-bed1-c36c7f1bdd55"). InnerVolumeSpecName "kube-api-access-72vg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:16:31.980615 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.980545    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47281a85-af87-4dc7-bed1-c36c7f1bdd55-extensions-socket-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:16:31.980615 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:31.980568    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72vg8\" (UniqueName: \"kubernetes.io/projected/47281a85-af87-4dc7-bed1-c36c7f1bdd55-kube-api-access-72vg8\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:16:32.288859 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:32.288785    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"
Jul 30 17:16:32.288985 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:32.288782    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc" event={"ID":"47281a85-af87-4dc7-bed1-c36c7f1bdd55","Type":"ContainerDied","Data":"24d2ae4d7206f95d355282be8c51a9e2fe3c2c3c7d9c5eb37406d3a0827fff82"}
Jul 30 17:16:32.288985 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:32.288917    2561 scope.go:117] "RemoveContainer" containerID="2306e5c48bb269e62172f8ba95e2c97e7bfb0b76a56b37080baeb55b0fb6a642"
Jul 30 17:16:32.310398 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:32.310376    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:16:32.315052 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:32.315030    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7xxc"]
Jul 30 17:16:33.340798 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:16:33.340762    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" path="/var/lib/kubelet/pods/47281a85-af87-4dc7-bed1-c36c7f1bdd55/volumes"
Jul 30 17:17:11.420000 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:11.419903    2561 scope.go:117] "RemoveContainer" containerID="9f1433a93770197e2d05db78029f095a5ba214fd068fd039bd8af43205e48485"
Jul 30 17:17:46.062138 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062104    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"]
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062476    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" containerName="manager"
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062501    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" containerName="manager"
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062522    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc0f022d-8a34-4997-9b68-046059a14a71" containerName="revoke-keys"
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062528    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc0f022d-8a34-4997-9b68-046059a14a71" containerName="revoke-keys"
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062578    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc0f022d-8a34-4997-9b68-046059a14a71" containerName="revoke-keys"
Jul 30 17:17:46.062629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.062588    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="47281a85-af87-4dc7-bed1-c36c7f1bdd55" containerName="manager"
Jul 30 17:17:46.065570 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.065543    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.068328 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.068307    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dcm8j\""
Jul 30 17:17:46.082355 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.082325    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"]
Jul 30 17:17:46.254705 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.254670    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6rn\" (UniqueName: \"kubernetes.io/projected/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-kube-api-access-tx6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.254872 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.254722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.355403 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.355366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6rn\" (UniqueName: \"kubernetes.io/projected/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-kube-api-access-tx6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.355593 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.355410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.355756 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.355741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.372563 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.372536    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6rn\" (UniqueName: \"kubernetes.io/projected/c8d3bcb5-21a4-4299-863a-a406d25ccc4a-kube-api-access-tx6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-rpbrq\" (UID: \"c8d3bcb5-21a4-4299-863a-a406d25ccc4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.375360 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.375335    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.526258 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.526232    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"]
Jul 30 17:17:46.527433 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:17:46.527411    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d3bcb5_21a4_4299_863a_a406d25ccc4a.slice/crio-1ade30d7f44614db7cce3504c442e49e7a37869698fad5e28f20db75d390573b WatchSource:0}: Error finding container 1ade30d7f44614db7cce3504c442e49e7a37869698fad5e28f20db75d390573b: Status 404 returned error can't find the container with id 1ade30d7f44614db7cce3504c442e49e7a37869698fad5e28f20db75d390573b
Jul 30 17:17:46.647601 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.647529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq" event={"ID":"c8d3bcb5-21a4-4299-863a-a406d25ccc4a","Type":"ContainerStarted","Data":"0dc7738a4401c0738fb4842618d98135c9ff14a12c3a70e1a981a1195ad864ba"}
Jul 30 17:17:46.647601 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.647588    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq" event={"ID":"c8d3bcb5-21a4-4299-863a-a406d25ccc4a","Type":"ContainerStarted","Data":"1ade30d7f44614db7cce3504c442e49e7a37869698fad5e28f20db75d390573b"}
Jul 30 17:17:46.647742 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.647658    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:17:46.673126 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:46.673078    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq" podStartSLOduration=0.673061571 podStartE2EDuration="673.061571ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:17:46.670628909 +0000 UTC m=+2495.928090350" watchObservedRunningTime="2026-07-30 17:17:46.673061571 +0000 UTC m=+2495.930522992"
Jul 30 17:17:57.654867 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:17:57.654832    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rpbrq"
Jul 30 17:18:07.928589 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:07.928555    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:18:11.441155 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:11.441110    2561 scope.go:117] "RemoveContainer" containerID="7a2b9d2e41d0355fda6e0ec95357bce193b1c33c2544f7af7155ec1c4cf19399"
Jul 30 17:18:11.449270 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:11.449251    2561 scope.go:117] "RemoveContainer" containerID="bc789928a58285ee39494acf0732e3475cbd563729e6e28ec1f6ba543c3c44a5"
Jul 30 17:18:18.017445 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:18.017408    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:18:45.524161 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.524127    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:18:45.527436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.527414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.530710 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.530680    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8eefd8d1\"/\"kube-root-ca.crt\""
Jul 30 17:18:45.530710 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.530692    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8eefd8d1\"/\"e2e-lifecycle-model-8eefd8d1-kserve-self-signed-certs\""
Jul 30 17:18:45.530710 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.530683    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8eefd8d1\"/\"openshift-service-ca.crt\""
Jul 30 17:18:45.530964 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.530760    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8eefd8d1\"/\"default-dockercfg-5xzvv\""
Jul 30 17:18:45.538242 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.538221    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:18:45.641900 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.641871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.641900 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.641904    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.642113 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.641953    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.642113 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.641979    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.642113 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.642005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.642113 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.642027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz582\" (UniqueName: \"kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743287 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743245    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743287 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743289    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743567 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743338    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743567 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743369    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743567 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743723 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz582\" (UniqueName: \"kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743723 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743699    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743833 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.743833 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.743791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.745673 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.745654    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.745873 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.745857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.751956 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.751930    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz582\" (UniqueName: \"kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582\") pod \"e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") " pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.837744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.837718    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:45.970533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:45.970477    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:18:45.973431 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:18:45.973385    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbc1a24e_2072_4b7b_bce0_ea2932e51893.slice/crio-48fcf01f03a349e5d904218a0bec4a12f4b2134ba0ad193c9fea448f74b23982 WatchSource:0}: Error finding container 48fcf01f03a349e5d904218a0bec4a12f4b2134ba0ad193c9fea448f74b23982: Status 404 returned error can't find the container with id 48fcf01f03a349e5d904218a0bec4a12f4b2134ba0ad193c9fea448f74b23982
Jul 30 17:18:46.932704 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:46.932665    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" event={"ID":"fbc1a24e-2072-4b7b-bce0-ea2932e51893","Type":"ContainerStarted","Data":"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"}
Jul 30 17:18:46.933117 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:46.932719    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" event={"ID":"fbc1a24e-2072-4b7b-bce0-ea2932e51893","Type":"ContainerStarted","Data":"48fcf01f03a349e5d904218a0bec4a12f4b2134ba0ad193c9fea448f74b23982"}
Jul 30 17:18:46.965427 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:46.965365    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" podStartSLOduration=1.965346117 podStartE2EDuration="1.965346117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:18:46.963277509 +0000 UTC m=+2556.220738931" watchObservedRunningTime="2026-07-30 17:18:46.965346117 +0000 UTC m=+2556.222807517"
Jul 30 17:18:48.714529 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:48.714481    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:18:55.838139 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:55.838104    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:55.838673 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:55.838304    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:55.850717 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:55.850697    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:18:55.989806 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:18:55.989782    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:19:11.471191 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:11.471155    2561 scope.go:117] "RemoveContainer" containerID="8bfb4998eee9bd2b57c46f59d49f3af5c7968530dbba310dc098af2471261c54"
Jul 30 17:19:20.013165 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:20.013125    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:19:32.549374 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.549297    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"]
Jul 30 17:19:32.552800 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.552780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:32.555602 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.555583    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:19:32.555689 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.555623    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:19:32.561981 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.561956    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"]
Jul 30 17:19:32.629035 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.628994    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw\" (UID: \"52fbde73-b2ca-4eef-9515-44e30b30b077\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:32.730220 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.730185    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw\" (UID: \"52fbde73-b2ca-4eef-9515-44e30b30b077\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:32.730783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.730765    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw\" (UID: \"52fbde73-b2ca-4eef-9515-44e30b30b077\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:32.863692 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.863665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:32.998242 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:32.998218    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"]
Jul 30 17:19:33.000786 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:19:33.000757    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52fbde73_b2ca_4eef_9515_44e30b30b077.slice/crio-53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366 WatchSource:0}: Error finding container 53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366: Status 404 returned error can't find the container with id 53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366
Jul 30 17:19:33.150646 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:33.150564    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw" event={"ID":"52fbde73-b2ca-4eef-9515-44e30b30b077","Type":"ContainerStarted","Data":"53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366"}
Jul 30 17:19:34.161612 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:34.161574    2561 generic.go:356] "Generic (PLEG): container finished" podID="52fbde73-b2ca-4eef-9515-44e30b30b077" containerID="d6721fc6e1702225d7e8bfbf3a35d916cf8da1b8753e47d06193eb03e9c9141b" exitCode=0
Jul 30 17:19:34.162015 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:34.161662    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw" event={"ID":"52fbde73-b2ca-4eef-9515-44e30b30b077","Type":"ContainerDied","Data":"d6721fc6e1702225d7e8bfbf3a35d916cf8da1b8753e47d06193eb03e9c9141b"}
Jul 30 17:19:35.307874 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:35.307846    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:35.455767 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:35.455676    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca\") pod \"52fbde73-b2ca-4eef-9515-44e30b30b077\" (UID: \"52fbde73-b2ca-4eef-9515-44e30b30b077\") "
Jul 30 17:19:35.456051 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:35.456028    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52fbde73-b2ca-4eef-9515-44e30b30b077" (UID: "52fbde73-b2ca-4eef-9515-44e30b30b077"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:19:35.556471 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:35.556440    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fbde73-b2ca-4eef-9515-44e30b30b077-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:19:36.171453 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:36.171426    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"
Jul 30 17:19:36.171646 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:36.171476    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw" event={"ID":"52fbde73-b2ca-4eef-9515-44e30b30b077","Type":"ContainerDied","Data":"53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366"}
Jul 30 17:19:36.171646 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:36.171548    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53414b90a9ecf4f32e1008e594809cd72317e76614229f3f126c28b5b4091366"
Jul 30 17:19:46.823135 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:46.823101    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:19:48.258645 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:48.258608    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"]
Jul 30 17:19:48.262908 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:48.262872    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8eefd8d1-kjqsw"]
Jul 30 17:19:49.341226 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:49.341187    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52fbde73-b2ca-4eef-9515-44e30b30b077" path="/var/lib/kubelet/pods/52fbde73-b2ca-4eef-9515-44e30b30b077/volumes"
Jul 30 17:19:56.528744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:56.528709    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:19:56.633296 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:19:56.633264    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:20:11.652582 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:11.652523    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" podUID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" containerName="main" containerID="cri-o://20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686" gracePeriod=45
Jul 30 17:20:11.906352 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:11.906295    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:20:12.057975 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.057947    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058162 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058022    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058162 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058050    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058162 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058089    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz582\" (UniqueName: \"kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058162 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058117    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058372 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058191    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir\") pod \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\" (UID: \"fbc1a24e-2072-4b7b-bce0-ea2932e51893\") "
Jul 30 17:20:12.058372 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058213    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home" (OuterVolumeSpecName: "home") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:20:12.058372 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache" (OuterVolumeSpecName: "model-cache") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:20:12.058568 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058458    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:20:12.058568 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058515    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.058568 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.058532    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.060360 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.060331    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm" (OuterVolumeSpecName: "dshm") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:20:12.060472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.060371    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:20:12.060472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.060391    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582" (OuterVolumeSpecName: "kube-api-access-kz582") pod "fbc1a24e-2072-4b7b-bce0-ea2932e51893" (UID: "fbc1a24e-2072-4b7b-bce0-ea2932e51893"). InnerVolumeSpecName "kube-api-access-kz582". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:20:12.159088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.159023    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.159088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.159046    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1a24e-2072-4b7b-bce0-ea2932e51893-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.159088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.159055    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz582\" (UniqueName: \"kubernetes.io/projected/fbc1a24e-2072-4b7b-bce0-ea2932e51893-kube-api-access-kz582\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.159088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.159063    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc1a24e-2072-4b7b-bce0-ea2932e51893-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:20:12.340203 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.340171    2561 generic.go:356] "Generic (PLEG): container finished" podID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" containerID="20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686" exitCode=0
Jul 30 17:20:12.340396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.340218    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"
Jul 30 17:20:12.340396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.340245    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" event={"ID":"fbc1a24e-2072-4b7b-bce0-ea2932e51893","Type":"ContainerDied","Data":"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"}
Jul 30 17:20:12.340396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.340276    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g" event={"ID":"fbc1a24e-2072-4b7b-bce0-ea2932e51893","Type":"ContainerDied","Data":"48fcf01f03a349e5d904218a0bec4a12f4b2134ba0ad193c9fea448f74b23982"}
Jul 30 17:20:12.340396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.340298    2561 scope.go:117] "RemoveContainer" containerID="20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"
Jul 30 17:20:12.353645 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.353624    2561 scope.go:117] "RemoveContainer" containerID="20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"
Jul 30 17:20:12.353936 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:20:12.353914    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686\": container with ID starting with 20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686 not found: ID does not exist" containerID="20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"
Jul 30 17:20:12.354005 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.353945    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686"} err="failed to get container status \"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686\": rpc error: code = NotFound desc = could not find container \"20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686\": container with ID starting with 20b88039aa6d8bdf377a3a01e2239fe23ef303e9b07f6a8894d4d3f5fd4d0686 not found: ID does not exist"
Jul 30 17:20:12.365428 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.365401    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:20:12.369365 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:12.369345    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8eefd8d1/e2e-lifecycle-model-8eefd8d1-kserve-784d49664c-xz42g"]
Jul 30 17:20:13.340501 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:13.340461    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" path="/var/lib/kubelet/pods/fbc1a24e-2072-4b7b-bce0-ea2932e51893/volumes"
Jul 30 17:20:42.630055 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:20:42.630014    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:21:06.220086 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:06.220050    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:21:11.367170 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:11.367141    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:21:11.373008 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:11.372981    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:21:11.374556 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:11.374533    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:21:11.380001 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:11.379983    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:21:13.580716 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:21:13.580680    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:22:04.320017 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:04.319979    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:22:11.507815 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:11.507718    2561 scope.go:117] "RemoveContainer" containerID="f70f3ecf094b987cdf58c53da2ffaa5ec70fac50906c1d216da99455e8e62dbe"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.585198    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586013    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" containerName="main"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586035    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" containerName="main"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586079    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52fbde73-b2ca-4eef-9515-44e30b30b077" containerName="revoke-keys"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586089    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="52fbde73-b2ca-4eef-9515-44e30b30b077" containerName="revoke-keys"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586282    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="52fbde73-b2ca-4eef-9515-44e30b30b077" containerName="revoke-keys"
Jul 30 17:22:30.587955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.586306    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc1a24e-2072-4b7b-bce0-ea2932e51893" containerName="main"
Jul 30 17:22:30.590595 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.590570    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.593533 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.593479    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4c7e9f-dockercfg-v8qc6\""
Jul 30 17:22:30.593652 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.593536    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4c7e9f\""
Jul 30 17:22:30.600437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.600413    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:22:30.620448 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.620424    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l99h\" (UniqueName: \"kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.620565 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.620541    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.721037 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.720995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l99h\" (UniqueName: \"kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.721203 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.721077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.721697 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.721681    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.729458 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.729433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l99h\" (UniqueName: \"kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h\") pod \"payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:30.902610 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:30.902532    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:31.035257 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:31.035231    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:22:31.037149 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:22:31.037121    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe06e3e0_2bca_40b9_96c1_1bc096a1329a.slice/crio-4e26a2ae726d62af294e7cd1e2d6da4e750e90b50bb43c0fd69b63d09bb33a93 WatchSource:0}: Error finding container 4e26a2ae726d62af294e7cd1e2d6da4e750e90b50bb43c0fd69b63d09bb33a93: Status 404 returned error can't find the container with id 4e26a2ae726d62af294e7cd1e2d6da4e750e90b50bb43c0fd69b63d09bb33a93
Jul 30 17:22:31.038925 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:31.038909    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:22:32.002414 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:32.002378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" event={"ID":"fe06e3e0-2bca-40b9-96c1-1bc096a1329a","Type":"ContainerStarted","Data":"4e26a2ae726d62af294e7cd1e2d6da4e750e90b50bb43c0fd69b63d09bb33a93"}
Jul 30 17:22:34.013479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:34.013437    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" event={"ID":"fe06e3e0-2bca-40b9-96c1-1bc096a1329a","Type":"ContainerStarted","Data":"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"}
Jul 30 17:22:34.013982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:34.013565    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:22:34.030413 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:34.030251    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" podStartSLOduration=2.115951863 podStartE2EDuration="4.030232046s" totalImagesPullingTime="1.914280183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:34.029459546 +0000 UTC m=+2783.286920987" watchObservedRunningTime="2026-07-30 17:22:34.030232046 +0000 UTC m=+2783.287693444"
Jul 30 17:22:35.123375 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:35.123342    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:22:45.021227 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:22:45.021198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:23:19.524711 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.524678    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:23:19.527971 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.527954    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.531460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.531341    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9f05e7\"/\"openshift-service-ca.crt\""
Jul 30 17:23:19.531460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.531354    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9f05e7\"/\"kube-root-ca.crt\""
Jul 30 17:23:19.531460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.531374    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9f05e7\"/\"default-dockercfg-spmrj\""
Jul 30 17:23:19.531706 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.531417    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9f05e7\"/\"test-model-9f05e7-kserve-self-signed-certs\""
Jul 30 17:23:19.536968 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.536946    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:23:19.617819 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.617819 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617829    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.617996 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617852    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.617996 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.617996 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617935    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.617996 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.617959    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t94hw\" (UniqueName: \"kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.718826 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.718797    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.718973 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.718835    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.718973 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.718853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719072 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719047    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719178 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t94hw\" (UniqueName: \"kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719397 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719242    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719397 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719259    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.719397 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.719376    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.721275 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.721257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.721348 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.721258    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.727669 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.727637    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t94hw\" (UniqueName: \"kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw\") pod \"test-model-9f05e7-kserve-54cc56bd69-6hz64\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") " pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.838793 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.838768    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:19.970062 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:19.970038    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:23:19.972598 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:23:19.972570    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod466b9d85_8fd8_4fbf_aede_dd54a3173efb.slice/crio-0cfd8820d7851d66567ad7e8e823cd834233d72026e1f7f471e40380c7d659ff WatchSource:0}: Error finding container 0cfd8820d7851d66567ad7e8e823cd834233d72026e1f7f471e40380c7d659ff: Status 404 returned error can't find the container with id 0cfd8820d7851d66567ad7e8e823cd834233d72026e1f7f471e40380c7d659ff
Jul 30 17:23:20.232121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:20.232026    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" event={"ID":"466b9d85-8fd8-4fbf-aede-dd54a3173efb","Type":"ContainerStarted","Data":"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"}
Jul 30 17:23:20.232121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:20.232071    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" event={"ID":"466b9d85-8fd8-4fbf-aede-dd54a3173efb","Type":"ContainerStarted","Data":"0cfd8820d7851d66567ad7e8e823cd834233d72026e1f7f471e40380c7d659ff"}
Jul 30 17:23:20.251426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:20.251375    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" podStartSLOduration=1.251341926 podStartE2EDuration="1.251341926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:23:20.249646705 +0000 UTC m=+2829.507108124" watchObservedRunningTime="2026-07-30 17:23:20.251341926 +0000 UTC m=+2829.508803344"
Jul 30 17:23:23.929318 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:23.929280    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:23:29.839344 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:29.839302    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:29.839877 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:29.839666    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:29.851745 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:29.851721    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:30.292417 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:30.292336    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:23:44.325607 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.325576    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:23:44.330107 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.330085    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.333598 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.333567    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4c7e9f\"/\"openshift-service-ca.crt\""
Jul 30 17:23:44.333598 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.333587    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4c7e9f\"/\"default-dockercfg-m26f9\""
Jul 30 17:23:44.333783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.333594    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4c7e9f\"/\"kube-root-ca.crt\""
Jul 30 17:23:44.333783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.333689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4c7e9f\"/\"test-model-4c7e9f-kserve-self-signed-certs\""
Jul 30 17:23:44.341190 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.341170    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:23:44.421938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.421908    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.422100 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.421978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxv2s\" (UniqueName: \"kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.422100 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.422009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.422100 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.422029    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.422100 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.422072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.422100 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.422096    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.522972 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.522931    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxv2s\" (UniqueName: \"kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.522986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523012    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523039    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523073    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523423 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523554 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523535    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523624 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523579    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.523661 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.523622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.525408 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.525391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.525676 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.525661    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.532784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.532761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxv2s\" (UniqueName: \"kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s\") pod \"test-model-4c7e9f-kserve-5d8cff8f77-rdlhb\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") " pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.640350 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.640320    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:44.780221 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:44.780070    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:23:44.783190 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:23:44.783159    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00f3219a_45fb_4c52_bf6f_360e1513bbf5.slice/crio-37424f1c5f359d74ca450129e399de0d863499c81ecdda9edbfe68a61dfd52ab WatchSource:0}: Error finding container 37424f1c5f359d74ca450129e399de0d863499c81ecdda9edbfe68a61dfd52ab: Status 404 returned error can't find the container with id 37424f1c5f359d74ca450129e399de0d863499c81ecdda9edbfe68a61dfd52ab
Jul 30 17:23:45.349296 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:45.349259    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" event={"ID":"00f3219a-45fb-4c52-bf6f-360e1513bbf5","Type":"ContainerStarted","Data":"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"}
Jul 30 17:23:45.349296 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:45.349302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" event={"ID":"00f3219a-45fb-4c52-bf6f-360e1513bbf5","Type":"ContainerStarted","Data":"37424f1c5f359d74ca450129e399de0d863499c81ecdda9edbfe68a61dfd52ab"}
Jul 30 17:23:45.366750 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:45.366703    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" podStartSLOduration=1.366686422 podStartE2EDuration="1.366686422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:23:45.365337358 +0000 UTC m=+2854.622798777" watchObservedRunningTime="2026-07-30 17:23:45.366686422 +0000 UTC m=+2854.624147841"
Jul 30 17:23:49.724353 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:49.724314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:23:54.640736 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:54.640703    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:54.641196 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:54.640752    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:54.653239 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:54.653209    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:23:55.406338 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:23:55.406304    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:24:24.045653 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:24.045622    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:24:25.017115 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.017071    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:24:25.189853 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.189816    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"]
Jul 30 17:24:25.193221 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.193203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:25.195649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.195626    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:24:25.195649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.195639    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:24:25.200644 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.200617    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"]
Jul 30 17:24:25.261791 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.261755    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj\" (UID: \"2de29cd0-0ed2-4803-b7c9-c3d476b4951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:25.362436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.362408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj\" (UID: \"2de29cd0-0ed2-4803-b7c9-c3d476b4951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:25.362990 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.362971    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj\" (UID: \"2de29cd0-0ed2-4803-b7c9-c3d476b4951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:25.504703 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.504660    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:25.639106 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:25.639084    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"]
Jul 30 17:24:25.641935 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:24:25.641901    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de29cd0_0ed2_4803_b7c9_c3d476b4951e.slice/crio-d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe WatchSource:0}: Error finding container d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe: Status 404 returned error can't find the container with id d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe
Jul 30 17:24:26.543356 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:26.543326    2561 generic.go:356] "Generic (PLEG): container finished" podID="2de29cd0-0ed2-4803-b7c9-c3d476b4951e" containerID="8a3eebcefe3169b9e88502ad9cf70a94096bd267dc7ffd9bd8764a1a6c4e4f4a" exitCode=0
Jul 30 17:24:26.543734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:26.543385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj" event={"ID":"2de29cd0-0ed2-4803-b7c9-c3d476b4951e","Type":"ContainerDied","Data":"8a3eebcefe3169b9e88502ad9cf70a94096bd267dc7ffd9bd8764a1a6c4e4f4a"}
Jul 30 17:24:26.543734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:26.543421    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj" event={"ID":"2de29cd0-0ed2-4803-b7c9-c3d476b4951e","Type":"ContainerStarted","Data":"d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe"}
Jul 30 17:24:27.675781 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:27.675758    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:27.780309 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:27.780281    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca\") pod \"2de29cd0-0ed2-4803-b7c9-c3d476b4951e\" (UID: \"2de29cd0-0ed2-4803-b7c9-c3d476b4951e\") "
Jul 30 17:24:27.780631 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:27.780609    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2de29cd0-0ed2-4803-b7c9-c3d476b4951e" (UID: "2de29cd0-0ed2-4803-b7c9-c3d476b4951e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:27.881524 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:27.881495    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de29cd0-0ed2-4803-b7c9-c3d476b4951e-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:28.553910 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:28.553879    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj" event={"ID":"2de29cd0-0ed2-4803-b7c9-c3d476b4951e","Type":"ContainerDied","Data":"d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe"}
Jul 30 17:24:28.553910 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:28.553893    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"
Jul 30 17:24:28.554185 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:28.553925    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0cc770d097b4e6b79b493416c38e1e011d541a25ca8c04c5f3f6f096a6383fe"
Jul 30 17:24:31.620853 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:31.620821    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:24:36.113822 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:36.113781    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"]
Jul 30 17:24:36.115287 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:36.115266    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9f05e7-gwvfj"]
Jul 30 17:24:37.340851 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:37.340814    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de29cd0-0ed2-4803-b7c9-c3d476b4951e" path="/var/lib/kubelet/pods/2de29cd0-0ed2-4803-b7c9-c3d476b4951e/volumes"
Jul 30 17:24:39.066843 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.066788    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" podUID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" containerName="main" containerID="cri-o://f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd" gracePeriod=45
Jul 30 17:24:39.315924 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.315902    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:24:39.382327 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382300    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382550 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382372    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382550 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382417    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382550 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382443    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382550 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382479    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t94hw\" (UniqueName: \"kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382572    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir\") pod \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\" (UID: \"466b9d85-8fd8-4fbf-aede-dd54a3173efb\") "
Jul 30 17:24:39.382784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382596    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home" (OuterVolumeSpecName: "home") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:39.382784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382672    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache" (OuterVolumeSpecName: "model-cache") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:39.382945 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382812    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.382945 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382829    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.382945 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.382881    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:39.384686 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.384653    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw" (OuterVolumeSpecName: "kube-api-access-t94hw") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "kube-api-access-t94hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:39.384815 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.384725    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm" (OuterVolumeSpecName: "dshm") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:39.384815 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.384734    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "466b9d85-8fd8-4fbf-aede-dd54a3173efb" (UID: "466b9d85-8fd8-4fbf-aede-dd54a3173efb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:24:39.483527 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.483481    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.483527 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.483523    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.483527 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.483534    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t94hw\" (UniqueName: \"kubernetes.io/projected/466b9d85-8fd8-4fbf-aede-dd54a3173efb-kube-api-access-t94hw\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.483726 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.483541    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/466b9d85-8fd8-4fbf-aede-dd54a3173efb-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:39.611240 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.611206    2561 generic.go:356] "Generic (PLEG): container finished" podID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" containerID="f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd" exitCode=0
Jul 30 17:24:39.611387 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.611248    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"
Jul 30 17:24:39.611387 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.611293    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" event={"ID":"466b9d85-8fd8-4fbf-aede-dd54a3173efb","Type":"ContainerDied","Data":"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"}
Jul 30 17:24:39.611387 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.611344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64" event={"ID":"466b9d85-8fd8-4fbf-aede-dd54a3173efb","Type":"ContainerDied","Data":"0cfd8820d7851d66567ad7e8e823cd834233d72026e1f7f471e40380c7d659ff"}
Jul 30 17:24:39.611387 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.611362    2561 scope.go:117] "RemoveContainer" containerID="f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"
Jul 30 17:24:39.620569 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.620553    2561 scope.go:117] "RemoveContainer" containerID="f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"
Jul 30 17:24:39.620837 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:24:39.620820    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd\": container with ID starting with f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd not found: ID does not exist" containerID="f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"
Jul 30 17:24:39.620896 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.620844    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd"} err="failed to get container status \"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd\": rpc error: code = NotFound desc = could not find container \"f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd\": container with ID starting with f3b50db201437dc40495c05f4d298318795cfe8b8a32b31d078fc93d074ce4dd not found: ID does not exist"
Jul 30 17:24:39.632741 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.632689    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:24:39.637091 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:39.637072    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9f05e7/test-model-9f05e7-kserve-54cc56bd69-6hz64"]
Jul 30 17:24:40.035938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.035838    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" podUID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" containerName="main" containerID="cri-o://390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b" gracePeriod=45
Jul 30 17:24:40.294347 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.294289    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:24:40.390008 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.389977    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390008 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390010    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390254 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390026    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390254 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390081    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390254 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390126    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxv2s\" (UniqueName: \"kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390254 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390157    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache\") pod \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\" (UID: \"00f3219a-45fb-4c52-bf6f-360e1513bbf5\") "
Jul 30 17:24:40.390472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390324    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home" (OuterVolumeSpecName: "home") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:40.390472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390424    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.390614 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390515    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache" (OuterVolumeSpecName: "model-cache") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:40.390614 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.390530    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:40.392354 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.392329    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm" (OuterVolumeSpecName: "dshm") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:24:40.392354 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.392332    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:24:40.392518 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.392331    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s" (OuterVolumeSpecName: "kube-api-access-jxv2s") pod "00f3219a-45fb-4c52-bf6f-360e1513bbf5" (UID: "00f3219a-45fb-4c52-bf6f-360e1513bbf5"). InnerVolumeSpecName "kube-api-access-jxv2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:40.491331 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.491295    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.491331 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.491326    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxv2s\" (UniqueName: \"kubernetes.io/projected/00f3219a-45fb-4c52-bf6f-360e1513bbf5-kube-api-access-jxv2s\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.491331 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.491336    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.491591 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.491344    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00f3219a-45fb-4c52-bf6f-360e1513bbf5-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.491591 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.491352    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00f3219a-45fb-4c52-bf6f-360e1513bbf5-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:40.624619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.624586    2561 generic.go:356] "Generic (PLEG): container finished" podID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" containerID="390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b" exitCode=0
Jul 30 17:24:40.624775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.624640    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"
Jul 30 17:24:40.624775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.624647    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" event={"ID":"00f3219a-45fb-4c52-bf6f-360e1513bbf5","Type":"ContainerDied","Data":"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"}
Jul 30 17:24:40.624775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.624677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb" event={"ID":"00f3219a-45fb-4c52-bf6f-360e1513bbf5","Type":"ContainerDied","Data":"37424f1c5f359d74ca450129e399de0d863499c81ecdda9edbfe68a61dfd52ab"}
Jul 30 17:24:40.624775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.624694    2561 scope.go:117] "RemoveContainer" containerID="390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"
Jul 30 17:24:40.634291 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.634273    2561 scope.go:117] "RemoveContainer" containerID="390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"
Jul 30 17:24:40.634560 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:24:40.634541    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b\": container with ID starting with 390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b not found: ID does not exist" containerID="390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"
Jul 30 17:24:40.634629 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.634566    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b"} err="failed to get container status \"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b\": rpc error: code = NotFound desc = could not find container \"390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b\": container with ID starting with 390de49a41d579845e0d1b63e7fb92b5f7b14b817cba725d8c1af26a3c746f5b not found: ID does not exist"
Jul 30 17:24:40.645970 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.645946    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:24:40.651639 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.651618    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4c7e9f/test-model-4c7e9f-kserve-5d8cff8f77-rdlhb"]
Jul 30 17:24:40.811307 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:40.811276    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:24:41.340351 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:41.340320    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" path="/var/lib/kubelet/pods/00f3219a-45fb-4c52-bf6f-360e1513bbf5/volumes"
Jul 30 17:24:41.340735 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:41.340708    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" path="/var/lib/kubelet/pods/466b9d85-8fd8-4fbf-aede-dd54a3173efb/volumes"
Jul 30 17:24:43.128861 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.128831    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"]
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129181    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" containerName="main"
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129191    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" containerName="main"
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129204    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de29cd0-0ed2-4803-b7c9-c3d476b4951e" containerName="revoke-keys"
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129210    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de29cd0-0ed2-4803-b7c9-c3d476b4951e" containerName="revoke-keys"
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129230    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" containerName="main"
Jul 30 17:24:43.129241 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129235    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" containerName="main"
Jul 30 17:24:43.129442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129288    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f3219a-45fb-4c52-bf6f-360e1513bbf5" containerName="main"
Jul 30 17:24:43.129442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129297    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="466b9d85-8fd8-4fbf-aede-dd54a3173efb" containerName="main"
Jul 30 17:24:43.129442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.129304    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de29cd0-0ed2-4803-b7c9-c3d476b4951e" containerName="revoke-keys"
Jul 30 17:24:43.133659 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.133643    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:43.136121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.136104    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:24:43.136210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.136103    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:24:43.139592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.139568    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"]
Jul 30 17:24:43.213699 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.213666    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx\" (UID: \"63fc0454-aa0b-4830-804a-36445d03fb83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:43.314759 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.314729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx\" (UID: \"63fc0454-aa0b-4830-804a-36445d03fb83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:43.315317 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.315294    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx\" (UID: \"63fc0454-aa0b-4830-804a-36445d03fb83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:43.444805 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.444730    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:43.576480 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.576453    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"]
Jul 30 17:24:43.578363 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:24:43.578335    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63fc0454_aa0b_4830_804a_36445d03fb83.slice/crio-cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158 WatchSource:0}: Error finding container cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158: Status 404 returned error can't find the container with id cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158
Jul 30 17:24:43.639374 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:43.639347    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx" event={"ID":"63fc0454-aa0b-4830-804a-36445d03fb83","Type":"ContainerStarted","Data":"cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158"}
Jul 30 17:24:44.646069 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:44.646027    2561 generic.go:356] "Generic (PLEG): container finished" podID="63fc0454-aa0b-4830-804a-36445d03fb83" containerID="c60546d5d3791e257c1f42f9f333fded71f44d8ef6069d3c378e8499f21639e6" exitCode=0
Jul 30 17:24:44.646441 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:44.646085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx" event={"ID":"63fc0454-aa0b-4830-804a-36445d03fb83","Type":"ContainerDied","Data":"c60546d5d3791e257c1f42f9f333fded71f44d8ef6069d3c378e8499f21639e6"}
Jul 30 17:24:45.783423 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:45.783396    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:45.837044 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:45.837022    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca\") pod \"63fc0454-aa0b-4830-804a-36445d03fb83\" (UID: \"63fc0454-aa0b-4830-804a-36445d03fb83\") "
Jul 30 17:24:45.837322 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:45.837301    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63fc0454-aa0b-4830-804a-36445d03fb83" (UID: "63fc0454-aa0b-4830-804a-36445d03fb83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:45.938421 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:45.938343    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fc0454-aa0b-4830-804a-36445d03fb83-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:46.658422 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:46.658392    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"
Jul 30 17:24:46.658422 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:46.658401    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx" event={"ID":"63fc0454-aa0b-4830-804a-36445d03fb83","Type":"ContainerDied","Data":"cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158"}
Jul 30 17:24:46.658650 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:46.658444    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf49e51f527328036c6624017b554258dc0a59cc18d7090a4f52b2e69df49158"
Jul 30 17:24:53.618770 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:53.618736    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:24:53.619265 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:53.618948    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" podUID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" containerName="payload-processing" containerID="cri-o://8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4" gracePeriod=30
Jul 30 17:24:53.779083 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:53.779044    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"]
Jul 30 17:24:53.783639 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:53.783614    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c7e9f-q5xjx"]
Jul 30 17:24:53.881892 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:53.881831    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:24:54.000069 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.000039    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume\") pod \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") "
Jul 30 17:24:54.000232 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.000117    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l99h\" (UniqueName: \"kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h\") pod \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\" (UID: \"fe06e3e0-2bca-40b9-96c1-1bc096a1329a\") "
Jul 30 17:24:54.000437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.000416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe06e3e0-2bca-40b9-96c1-1bc096a1329a" (UID: "fe06e3e0-2bca-40b9-96c1-1bc096a1329a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:54.002109 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.002087    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h" (OuterVolumeSpecName: "kube-api-access-5l99h") pod "fe06e3e0-2bca-40b9-96c1-1bc096a1329a" (UID: "fe06e3e0-2bca-40b9-96c1-1bc096a1329a"). InnerVolumeSpecName "kube-api-access-5l99h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:54.101112 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.101082    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l99h\" (UniqueName: \"kubernetes.io/projected/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-kube-api-access-5l99h\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.101112 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.101105    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e3e0-2bca-40b9-96c1-1bc096a1329a-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.702454 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.702422    2561 generic.go:356] "Generic (PLEG): container finished" podID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" containerID="8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4" exitCode=0
Jul 30 17:24:54.702875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.702466    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"
Jul 30 17:24:54.702875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.702511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" event={"ID":"fe06e3e0-2bca-40b9-96c1-1bc096a1329a","Type":"ContainerDied","Data":"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"}
Jul 30 17:24:54.702875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.702568    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x" event={"ID":"fe06e3e0-2bca-40b9-96c1-1bc096a1329a","Type":"ContainerDied","Data":"4e26a2ae726d62af294e7cd1e2d6da4e750e90b50bb43c0fd69b63d09bb33a93"}
Jul 30 17:24:54.702875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.702592    2561 scope.go:117] "RemoveContainer" containerID="8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"
Jul 30 17:24:54.711430 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.711408    2561 scope.go:117] "RemoveContainer" containerID="8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"
Jul 30 17:24:54.711715 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:24:54.711696    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4\": container with ID starting with 8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4 not found: ID does not exist" containerID="8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"
Jul 30 17:24:54.711851 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.711725    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4"} err="failed to get container status \"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4\": rpc error: code = NotFound desc = could not find container \"8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4\": container with ID starting with 8055e92cfb4f3aac4fba786b5a002e9f3c8b01168c8cf76c5bd4910c0c81edf4 not found: ID does not exist"
Jul 30 17:24:54.723367 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.723346    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:24:54.727437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:54.727418    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c7e9f-64f57bb5d8-ppx7x"]
Jul 30 17:24:55.340259 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:55.340226    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63fc0454-aa0b-4830-804a-36445d03fb83" path="/var/lib/kubelet/pods/63fc0454-aa0b-4830-804a-36445d03fb83/volumes"
Jul 30 17:24:55.340610 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:55.340597    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" path="/var/lib/kubelet/pods/fe06e3e0-2bca-40b9-96c1-1bc096a1329a/volumes"
Jul 30 17:24:56.620320 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:24:56.620284    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:25:06.820318 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820286    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:25:06.820719 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820634    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" containerName="payload-processing"
Jul 30 17:25:06.820719 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820646    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" containerName="payload-processing"
Jul 30 17:25:06.820719 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820665    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63fc0454-aa0b-4830-804a-36445d03fb83" containerName="revoke-keys"
Jul 30 17:25:06.820719 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820671    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="63fc0454-aa0b-4830-804a-36445d03fb83" containerName="revoke-keys"
Jul 30 17:25:06.820849 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820727    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="63fc0454-aa0b-4830-804a-36445d03fb83" containerName="revoke-keys"
Jul 30 17:25:06.820849 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.820735    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe06e3e0-2bca-40b9-96c1-1bc096a1329a" containerName="payload-processing"
Jul 30 17:25:06.825428 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.825407    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:06.828313 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.828294    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4dfe39-dockercfg-ns8rb\""
Jul 30 17:25:06.828600 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.828586    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4dfe39\""
Jul 30 17:25:06.834817 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.834795    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:25:06.909890 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.909853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:06.910101 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:06.909905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwsj\" (UniqueName: \"kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.011097 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.011054    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.011097 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.011099    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grwsj\" (UniqueName: \"kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.011809 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.011789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.020602 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.020578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwsj\" (UniqueName: \"kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj\") pod \"payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.137682 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.137639    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.284606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.284577    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:25:07.287153 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:25:07.287124    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf16bf4_cae2_4ae7_a1ff_0caf738a183c.slice/crio-cbbaaf4974b2874753e988034eb3f2a42ff2f3cb31124cd3976cbb1de5937dde WatchSource:0}: Error finding container cbbaaf4974b2874753e988034eb3f2a42ff2f3cb31124cd3976cbb1de5937dde: Status 404 returned error can't find the container with id cbbaaf4974b2874753e988034eb3f2a42ff2f3cb31124cd3976cbb1de5937dde
Jul 30 17:25:07.770141 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.770108    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" event={"ID":"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c","Type":"ContainerStarted","Data":"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"}
Jul 30 17:25:07.770351 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.770159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" event={"ID":"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c","Type":"ContainerStarted","Data":"cbbaaf4974b2874753e988034eb3f2a42ff2f3cb31124cd3976cbb1de5937dde"}
Jul 30 17:25:07.770351 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.770240    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:07.810884 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:07.810819    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" podStartSLOduration=1.810798648 podStartE2EDuration="1.810798648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:07.809826908 +0000 UTC m=+2937.067288345" watchObservedRunningTime="2026-07-30 17:25:07.810798648 +0000 UTC m=+2937.068260065"
Jul 30 17:25:11.814580 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:11.814498    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:25:18.776918 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:18.776886    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:25:35.522875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.522799    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:25:35.526283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.526267    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.528780 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.528758    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b9fcce-dockercfg-9ssbb\""
Jul 30 17:25:35.528933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.528766    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b9fcce\""
Jul 30 17:25:35.531814 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.531788    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfj7j\" (UniqueName: \"kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.531977 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.531948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.534877 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.534856    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:25:35.544788 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.544764    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:25:35.547848 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.547833    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.558796 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.558774    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:25:35.633213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.633181    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.633378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.633225    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.633378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.633296    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj7j\" (UniqueName: \"kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.633378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.633368    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfmj6\" (UniqueName: \"kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.633926 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.633909    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.644850 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.644827    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfj7j\" (UniqueName: \"kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j\") pod \"payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.734154 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.734128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfmj6\" (UniqueName: \"kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.734289 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.734176    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.734803 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.734785    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.742984 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.742962    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfmj6\" (UniqueName: \"kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6\") pod \"payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.837517 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.837480    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:35.858802 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.858775    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:35.989452 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:35.989426    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:25:35.991676 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:25:35.991652    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d839895_880d_40c0_9cfb_8127bc185dec.slice/crio-a88e16506334bdb6cecb81fa24e85f5d0a43e0c9c572cd5332f44fec2ef0fcd3 WatchSource:0}: Error finding container a88e16506334bdb6cecb81fa24e85f5d0a43e0c9c572cd5332f44fec2ef0fcd3: Status 404 returned error can't find the container with id a88e16506334bdb6cecb81fa24e85f5d0a43e0c9c572cd5332f44fec2ef0fcd3
Jul 30 17:25:36.019927 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.019905    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:25:36.022605 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:25:36.022583    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d7608b8_8e7f_4219_a38a_ad9b5404ce10.slice/crio-6c91a9725870cfc60e4ddbb1854e63423bf6708dd2c19a720e454f3359ed4a30 WatchSource:0}: Error finding container 6c91a9725870cfc60e4ddbb1854e63423bf6708dd2c19a720e454f3359ed4a30: Status 404 returned error can't find the container with id 6c91a9725870cfc60e4ddbb1854e63423bf6708dd2c19a720e454f3359ed4a30
Jul 30 17:25:36.917594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.917556    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" event={"ID":"1d7608b8-8e7f-4219-a38a-ad9b5404ce10","Type":"ContainerStarted","Data":"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"}
Jul 30 17:25:36.918071 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.917649    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:36.918071 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.917672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" event={"ID":"1d7608b8-8e7f-4219-a38a-ad9b5404ce10","Type":"ContainerStarted","Data":"6c91a9725870cfc60e4ddbb1854e63423bf6708dd2c19a720e454f3359ed4a30"}
Jul 30 17:25:36.919082 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.919055    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" event={"ID":"8d839895-880d-40c0-9cfb-8127bc185dec","Type":"ContainerStarted","Data":"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"}
Jul 30 17:25:36.919238 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.919099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" event={"ID":"8d839895-880d-40c0-9cfb-8127bc185dec","Type":"ContainerStarted","Data":"a88e16506334bdb6cecb81fa24e85f5d0a43e0c9c572cd5332f44fec2ef0fcd3"}
Jul 30 17:25:36.919238 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.919119    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:25:36.935622 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.935578    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" podStartSLOduration=1.9355638640000001 podStartE2EDuration="1.935563864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:36.933695025 +0000 UTC m=+2966.191156446" watchObservedRunningTime="2026-07-30 17:25:36.935563864 +0000 UTC m=+2966.193025262"
Jul 30 17:25:36.949881 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:36.949833    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" podStartSLOduration=1.949819745 podStartE2EDuration="1.949819745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:36.948092728 +0000 UTC m=+2966.205554147" watchObservedRunningTime="2026-07-30 17:25:36.949819745 +0000 UTC m=+2966.207281221"
Jul 30 17:25:40.126938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:40.126903    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:25:47.926870 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:47.926839    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:25:47.927257 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:25:47.926921    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:26:01.976329 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:01.976298    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"]
Jul 30 17:26:01.979676 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:01.979660    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:01.982272 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:01.982247    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:26:01.982387 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:01.982315    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:26:01.988981 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:01.988957    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"]
Jul 30 17:26:02.047640 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:02.047610    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv\" (UID: \"23bcd367-c4fe-4630-956d-3dadb3ab49b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:02.148900 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:02.148871    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv\" (UID: \"23bcd367-c4fe-4630-956d-3dadb3ab49b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:02.149467 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:02.149448    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv\" (UID: \"23bcd367-c4fe-4630-956d-3dadb3ab49b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:02.290813 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:02.290733    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:02.430673 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:02.430612    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"]
Jul 30 17:26:02.433012 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:26:02.432981    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bcd367_c4fe_4630_956d_3dadb3ab49b3.slice/crio-c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0 WatchSource:0}: Error finding container c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0: Status 404 returned error can't find the container with id c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0
Jul 30 17:26:03.055244 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:03.055152    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv" event={"ID":"23bcd367-c4fe-4630-956d-3dadb3ab49b3","Type":"ContainerStarted","Data":"7c8ea8f96e358433f2e18446190db17d20addbdf73e127e52ecfac88f6da5b62"}
Jul 30 17:26:03.055244 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:03.055220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv" event={"ID":"23bcd367-c4fe-4630-956d-3dadb3ab49b3","Type":"ContainerStarted","Data":"c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0"}
Jul 30 17:26:03.070425 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:03.070379    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv" podStartSLOduration=2.07036361 podStartE2EDuration="2.07036361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:03.069656478 +0000 UTC m=+2992.327117900" watchObservedRunningTime="2026-07-30 17:26:03.07036361 +0000 UTC m=+2992.327825391"
Jul 30 17:26:04.062298 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:04.062262    2561 generic.go:356] "Generic (PLEG): container finished" podID="23bcd367-c4fe-4630-956d-3dadb3ab49b3" containerID="7c8ea8f96e358433f2e18446190db17d20addbdf73e127e52ecfac88f6da5b62" exitCode=0
Jul 30 17:26:04.062729 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:04.062358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv" event={"ID":"23bcd367-c4fe-4630-956d-3dadb3ab49b3","Type":"ContainerDied","Data":"7c8ea8f96e358433f2e18446190db17d20addbdf73e127e52ecfac88f6da5b62"}
Jul 30 17:26:05.210618 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:05.210594    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:05.272668 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:05.272626    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca\") pod \"23bcd367-c4fe-4630-956d-3dadb3ab49b3\" (UID: \"23bcd367-c4fe-4630-956d-3dadb3ab49b3\") "
Jul 30 17:26:05.272954 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:05.272934    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23bcd367-c4fe-4630-956d-3dadb3ab49b3" (UID: "23bcd367-c4fe-4630-956d-3dadb3ab49b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:05.373732 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:05.373702    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23bcd367-c4fe-4630-956d-3dadb3ab49b3-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:06.074523 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:06.074495    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"
Jul 30 17:26:06.074708 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:06.074534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv" event={"ID":"23bcd367-c4fe-4630-956d-3dadb3ab49b3","Type":"ContainerDied","Data":"c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0"}
Jul 30 17:26:06.074708 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:06.074593    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4ef524b0cc37735a16e6ea9a1a71ae5cc7896a714e033897867aa32eecc1ab0"
Jul 30 17:26:11.396778 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:11.396752    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:26:11.402699 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:11.402680    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:26:11.403680 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:11.403659    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:26:11.408892 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:11.408877    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:26:11.555716 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:11.555693    2561 scope.go:117] "RemoveContainer" containerID="d6721fc6e1702225d7e8bfbf3a35d916cf8da1b8753e47d06193eb03e9c9141b"
Jul 30 17:26:17.572329 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.572297    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:26:17.572840 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.572602    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" podUID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" containerName="payload-pre-processing" containerID="cri-o://830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215" gracePeriod=30
Jul 30 17:26:17.730900 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.730858    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"]
Jul 30 17:26:17.734124 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.734089    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4dfe39-hgfgv"]
Jul 30 17:26:17.828814 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.828797    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:26:17.981503 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.981451    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grwsj\" (UniqueName: \"kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj\") pod \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") "
Jul 30 17:26:17.981658 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.981571    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume\") pod \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\" (UID: \"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c\") "
Jul 30 17:26:17.981961 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.981930    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" (UID: "0cf16bf4-cae2-4ae7-a1ff-0caf738a183c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:17.983445 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:17.983412    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj" (OuterVolumeSpecName: "kube-api-access-grwsj") pod "0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" (UID: "0cf16bf4-cae2-4ae7-a1ff-0caf738a183c"). InnerVolumeSpecName "kube-api-access-grwsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:18.083088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.083025    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:18.083088 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.083050    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grwsj\" (UniqueName: \"kubernetes.io/projected/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c-kube-api-access-grwsj\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:18.134757 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.134729    2561 generic.go:356] "Generic (PLEG): container finished" podID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" containerID="830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215" exitCode=0
Jul 30 17:26:18.134864 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.134778    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"
Jul 30 17:26:18.134864 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.134815    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" event={"ID":"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c","Type":"ContainerDied","Data":"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"}
Jul 30 17:26:18.134939 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.134900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw" event={"ID":"0cf16bf4-cae2-4ae7-a1ff-0caf738a183c","Type":"ContainerDied","Data":"cbbaaf4974b2874753e988034eb3f2a42ff2f3cb31124cd3976cbb1de5937dde"}
Jul 30 17:26:18.134939 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.134928    2561 scope.go:117] "RemoveContainer" containerID="830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"
Jul 30 17:26:18.143425 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.143408    2561 scope.go:117] "RemoveContainer" containerID="830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"
Jul 30 17:26:18.143706 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:26:18.143688    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215\": container with ID starting with 830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215 not found: ID does not exist" containerID="830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"
Jul 30 17:26:18.143775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.143717    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215"} err="failed to get container status \"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215\": rpc error: code = NotFound desc = could not find container \"830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215\": container with ID starting with 830f9968ab8f0e31e30924a04409369faae653b3afa39e8a134adf6825bc7215 not found: ID does not exist"
Jul 30 17:26:18.157808 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.157789    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:26:18.163470 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:18.163445    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4dfe39-6448ff968-fxtsw"]
Jul 30 17:26:19.340248 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:19.340216    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" path="/var/lib/kubelet/pods/0cf16bf4-cae2-4ae7-a1ff-0caf738a183c/volumes"
Jul 30 17:26:19.340636 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:19.340545    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23bcd367-c4fe-4630-956d-3dadb3ab49b3" path="/var/lib/kubelet/pods/23bcd367-c4fe-4630-956d-3dadb3ab49b3/volumes"
Jul 30 17:26:23.621215 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:23.621183    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:26:24.714759 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.714720    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"]
Jul 30 17:26:24.715245 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715224    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23bcd367-c4fe-4630-956d-3dadb3ab49b3" containerName="revoke-keys"
Jul 30 17:26:24.715337 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715249    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="23bcd367-c4fe-4630-956d-3dadb3ab49b3" containerName="revoke-keys"
Jul 30 17:26:24.715337 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715274    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" containerName="payload-pre-processing"
Jul 30 17:26:24.715337 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715284    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" containerName="payload-pre-processing"
Jul 30 17:26:24.715520 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715394    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf16bf4-cae2-4ae7-a1ff-0caf738a183c" containerName="payload-pre-processing"
Jul 30 17:26:24.715520 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.715411    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="23bcd367-c4fe-4630-956d-3dadb3ab49b3" containerName="revoke-keys"
Jul 30 17:26:24.718587 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.718567    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:24.721329 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.721306    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:26:24.721460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.721339    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:26:24.731116 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.731095    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"]
Jul 30 17:26:24.839818 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.839789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf\" (UID: \"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:24.940373 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.940342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf\" (UID: \"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:24.940971 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:24.940952    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf\" (UID: \"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:25.028598 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:25.028523    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:25.161963 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:25.161931    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"]
Jul 30 17:26:25.164667 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:26:25.164639    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3033e9_c0e7_4ddb_a826_3cdffeb41e2b.slice/crio-a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217 WatchSource:0}: Error finding container a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217: Status 404 returned error can't find the container with id a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217
Jul 30 17:26:25.172074 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:25.172053    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf" event={"ID":"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b","Type":"ContainerStarted","Data":"a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217"}
Jul 30 17:26:26.178579 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:26.178475    2561 generic.go:356] "Generic (PLEG): container finished" podID="de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" containerID="7b86cfae37c45e797d8c89e7e23c360fa2c2fa3d4b15b0482ebf0b0b2a6db819" exitCode=0
Jul 30 17:26:26.178579 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:26.178515    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf" event={"ID":"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b","Type":"ContainerDied","Data":"7b86cfae37c45e797d8c89e7e23c360fa2c2fa3d4b15b0482ebf0b0b2a6db819"}
Jul 30 17:26:27.332448 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:27.332425    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:27.464035 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:27.463936    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca\") pod \"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b\" (UID: \"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b\") "
Jul 30 17:26:27.464281 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:27.464255    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" (UID: "de3033e9-c0e7-4ddb-a826-3cdffeb41e2b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:27.565426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:27.565395    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:28.192270 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:28.192236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf" event={"ID":"de3033e9-c0e7-4ddb-a826-3cdffeb41e2b","Type":"ContainerDied","Data":"a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217"}
Jul 30 17:26:28.192270 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:28.192253    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"
Jul 30 17:26:28.192472 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:28.192283    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8026b458650afd1d5534744c6cf8c14660639a84b50b8fe88724ee44143a217"
Jul 30 17:26:40.387726 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.387687    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:26:40.388322 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.387951    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" podUID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" containerName="payload-processing" containerID="cri-o://813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3" gracePeriod=30
Jul 30 17:26:40.398864 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.398834    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:26:40.399130 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.399060    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" podUID="8d839895-880d-40c0-9cfb-8127bc185dec" containerName="payload-pre-processing" containerID="cri-o://a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211" gracePeriod=30
Jul 30 17:26:40.591776 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.591723    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"]
Jul 30 17:26:40.593235 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.593209    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9fcce-jxjjf"]
Jul 30 17:26:40.681691 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.681658    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:26:40.685147 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.685126    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:26:40.766396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766362    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume\") pod \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") "
Jul 30 17:26:40.766592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766414    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfmj6\" (UniqueName: \"kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6\") pod \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\" (UID: \"1d7608b8-8e7f-4219-a38a-ad9b5404ce10\") "
Jul 30 17:26:40.766592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766515    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume\") pod \"8d839895-880d-40c0-9cfb-8127bc185dec\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") "
Jul 30 17:26:40.766592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766540    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfj7j\" (UniqueName: \"kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j\") pod \"8d839895-880d-40c0-9cfb-8127bc185dec\" (UID: \"8d839895-880d-40c0-9cfb-8127bc185dec\") "
Jul 30 17:26:40.766862 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766834    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d7608b8-8e7f-4219-a38a-ad9b5404ce10" (UID: "1d7608b8-8e7f-4219-a38a-ad9b5404ce10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:40.767045 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.766874    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d839895-880d-40c0-9cfb-8127bc185dec" (UID: "8d839895-880d-40c0-9cfb-8127bc185dec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:40.769153 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.769129    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j" (OuterVolumeSpecName: "kube-api-access-pfj7j") pod "8d839895-880d-40c0-9cfb-8127bc185dec" (UID: "8d839895-880d-40c0-9cfb-8127bc185dec"). InnerVolumeSpecName "kube-api-access-pfj7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:40.769224 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.769138    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6" (OuterVolumeSpecName: "kube-api-access-xfmj6") pod "1d7608b8-8e7f-4219-a38a-ad9b5404ce10" (UID: "1d7608b8-8e7f-4219-a38a-ad9b5404ce10"). InnerVolumeSpecName "kube-api-access-xfmj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:40.867679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.867649    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d839895-880d-40c0-9cfb-8127bc185dec-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:40.867679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.867670    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfj7j\" (UniqueName: \"kubernetes.io/projected/8d839895-880d-40c0-9cfb-8127bc185dec-kube-api-access-pfj7j\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:40.867679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.867679    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:40.867679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:40.867688    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfmj6\" (UniqueName: \"kubernetes.io/projected/1d7608b8-8e7f-4219-a38a-ad9b5404ce10-kube-api-access-xfmj6\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:26:41.254631 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.254599    2561 generic.go:356] "Generic (PLEG): container finished" podID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" containerID="813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3" exitCode=0
Jul 30 17:26:41.254817 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.254639    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"
Jul 30 17:26:41.254817 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.254683    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" event={"ID":"1d7608b8-8e7f-4219-a38a-ad9b5404ce10","Type":"ContainerDied","Data":"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"}
Jul 30 17:26:41.254817 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.254749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc" event={"ID":"1d7608b8-8e7f-4219-a38a-ad9b5404ce10","Type":"ContainerDied","Data":"6c91a9725870cfc60e4ddbb1854e63423bf6708dd2c19a720e454f3359ed4a30"}
Jul 30 17:26:41.254817 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.254777    2561 scope.go:117] "RemoveContainer" containerID="813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"
Jul 30 17:26:41.256220 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.256193    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d839895-880d-40c0-9cfb-8127bc185dec" containerID="a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211" exitCode=0
Jul 30 17:26:41.256333 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.256231    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"
Jul 30 17:26:41.256333 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.256250    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" event={"ID":"8d839895-880d-40c0-9cfb-8127bc185dec","Type":"ContainerDied","Data":"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"}
Jul 30 17:26:41.256333 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.256289    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9" event={"ID":"8d839895-880d-40c0-9cfb-8127bc185dec","Type":"ContainerDied","Data":"a88e16506334bdb6cecb81fa24e85f5d0a43e0c9c572cd5332f44fec2ef0fcd3"}
Jul 30 17:26:41.264365 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.264237    2561 scope.go:117] "RemoveContainer" containerID="813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"
Jul 30 17:26:41.264642 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:26:41.264618    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3\": container with ID starting with 813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3 not found: ID does not exist" containerID="813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"
Jul 30 17:26:41.264739 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.264696    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3"} err="failed to get container status \"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3\": rpc error: code = NotFound desc = could not find container \"813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3\": container with ID starting with 813baeffd7e4a57ca0302e7c91503ca473925c8f788733dc7b5985ba5fc91da3 not found: ID does not exist"
Jul 30 17:26:41.264739 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.264725    2561 scope.go:117] "RemoveContainer" containerID="a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"
Jul 30 17:26:41.272953 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.272932    2561 scope.go:117] "RemoveContainer" containerID="a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"
Jul 30 17:26:41.273210 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:26:41.273195    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211\": container with ID starting with a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211 not found: ID does not exist" containerID="a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"
Jul 30 17:26:41.273268 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.273216    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211"} err="failed to get container status \"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211\": rpc error: code = NotFound desc = could not find container \"a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211\": container with ID starting with a034354c4e581c0bd0206dfeed93d143b3774c2876e7dacb52201f907a0c0211 not found: ID does not exist"
Jul 30 17:26:41.293085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.293064    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:26:41.306639 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.306614    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9fcce-645c669c6-5h5m9"]
Jul 30 17:26:41.323960 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.323935    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:26:41.340977 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.340951    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d839895-880d-40c0-9cfb-8127bc185dec" path="/var/lib/kubelet/pods/8d839895-880d-40c0-9cfb-8127bc185dec/volumes"
Jul 30 17:26:41.341325 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.341306    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" path="/var/lib/kubelet/pods/de3033e9-c0e7-4ddb-a826-3cdffeb41e2b/volumes"
Jul 30 17:26:41.341613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:41.341598    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9fcce-7dd78d978-hw4pc"]
Jul 30 17:26:43.341235 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:43.341192    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" path="/var/lib/kubelet/pods/1d7608b8-8e7f-4219-a38a-ad9b5404ce10/volumes"
Jul 30 17:26:45.037156 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:45.037123    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:26:54.414930 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.414895    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415230    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" containerName="payload-processing"
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415242    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" containerName="payload-processing"
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415254    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d839895-880d-40c0-9cfb-8127bc185dec" containerName="payload-pre-processing"
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415260    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d839895-880d-40c0-9cfb-8127bc185dec" containerName="payload-pre-processing"
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415267    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" containerName="revoke-keys"
Jul 30 17:26:54.415341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415272    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" containerName="revoke-keys"
Jul 30 17:26:54.415606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415345    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d839895-880d-40c0-9cfb-8127bc185dec" containerName="payload-pre-processing"
Jul 30 17:26:54.415606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415356    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d7608b8-8e7f-4219-a38a-ad9b5404ce10" containerName="payload-processing"
Jul 30 17:26:54.415606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.415363    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3033e9-c0e7-4ddb-a826-3cdffeb41e2b" containerName="revoke-keys"
Jul 30 17:26:54.418397 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.418378    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.421310 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.421290    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-f45123-dockercfg-z5tk9\""
Jul 30 17:26:54.421555 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.421539    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-f45123\""
Jul 30 17:26:54.430532 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.430511    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:26:54.462683 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.462652    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgx6d\" (UniqueName: \"kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.462811 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.462699    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.563361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.563322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.563558 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.563437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgx6d\" (UniqueName: \"kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.564218 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.564187    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.577998 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.577968    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgx6d\" (UniqueName: \"kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d\") pod \"payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:54.728353 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:54.728277    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:55.073970 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:55.073894    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:26:55.077098 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:26:55.077070    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dda4bbe_fb6c_4833_b759_efe57cfdbc46.slice/crio-ddf090d2416603bfd715252b1dcd19580c1e0906753061fc51b8dc24e6f3c15a WatchSource:0}: Error finding container ddf090d2416603bfd715252b1dcd19580c1e0906753061fc51b8dc24e6f3c15a: Status 404 returned error can't find the container with id ddf090d2416603bfd715252b1dcd19580c1e0906753061fc51b8dc24e6f3c15a
Jul 30 17:26:55.330213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:55.330180    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" event={"ID":"9dda4bbe-fb6c-4833-b759-efe57cfdbc46","Type":"ContainerStarted","Data":"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"}
Jul 30 17:26:55.330359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:55.330245    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:26:55.330359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:55.330262    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" event={"ID":"9dda4bbe-fb6c-4833-b759-efe57cfdbc46","Type":"ContainerStarted","Data":"ddf090d2416603bfd715252b1dcd19580c1e0906753061fc51b8dc24e6f3c15a"}
Jul 30 17:26:55.352963 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:55.352909    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" podStartSLOduration=1.352888065 podStartE2EDuration="1.352888065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:55.351890015 +0000 UTC m=+3044.609351434" watchObservedRunningTime="2026-07-30 17:26:55.352888065 +0000 UTC m=+3044.610349486"
Jul 30 17:26:58.829621 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:26:58.829582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:27:06.337945 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:06.337914    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:27:23.391598 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.391565    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:27:23.394737 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.394722    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.397647 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.397622    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-27deb9-dockercfg-nlwbh\""
Jul 30 17:27:23.397647 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.397645    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-27deb9\""
Jul 30 17:27:23.406300 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.406279    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:27:23.450679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.450654    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:27:23.453818 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.453803    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.463125 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.463106    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:27:23.503614 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.503589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.503744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.503626    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9lmf\" (UniqueName: \"kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.503744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.503657    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.503744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.503739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9t4f\" (UniqueName: \"kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.604628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.604591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.604789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.604644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9lmf\" (UniqueName: \"kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.604789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.604692    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.604789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.604730    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9t4f\" (UniqueName: \"kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.605376 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.605354    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.605376 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.605368    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.617006 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.616985    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9lmf\" (UniqueName: \"kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf\") pod \"payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.618438 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.618422    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9t4f\" (UniqueName: \"kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f\") pod \"payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.705723 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.705630    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:23.769702 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.769668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:23.910175 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:23.907791    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:27:23.910749 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:27:23.910722    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e8d615_a180_4053_8431_e12fa57229c6.slice/crio-d40f0ad5c93f0b7b92dfed3e16481d74d9ba2651d3c50d817c9e96a0573a735b WatchSource:0}: Error finding container d40f0ad5c93f0b7b92dfed3e16481d74d9ba2651d3c50d817c9e96a0573a735b: Status 404 returned error can't find the container with id d40f0ad5c93f0b7b92dfed3e16481d74d9ba2651d3c50d817c9e96a0573a735b
Jul 30 17:27:24.053714 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.053689    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:27:24.055640 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:27:24.055613    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b0103e_aed1_4ce0_8c7c_234f7563100f.slice/crio-d9d3b089628d9ccde40b22b8cb36a116bf9e18030fb5e27d8929f31c46cff506 WatchSource:0}: Error finding container d9d3b089628d9ccde40b22b8cb36a116bf9e18030fb5e27d8929f31c46cff506: Status 404 returned error can't find the container with id d9d3b089628d9ccde40b22b8cb36a116bf9e18030fb5e27d8929f31c46cff506
Jul 30 17:27:24.476190 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.476146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" event={"ID":"93e8d615-a180-4053-8431-e12fa57229c6","Type":"ContainerStarted","Data":"14ae4217c588a6b2b8626102ab866cb1a0ef1fc84e8befc139a061fd243f1c2f"}
Jul 30 17:27:24.476706 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.476214    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" event={"ID":"93e8d615-a180-4053-8431-e12fa57229c6","Type":"ContainerStarted","Data":"d40f0ad5c93f0b7b92dfed3e16481d74d9ba2651d3c50d817c9e96a0573a735b"}
Jul 30 17:27:24.476706 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.476242    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:24.477761 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.477736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" event={"ID":"d9b0103e-aed1-4ce0-8c7c-234f7563100f","Type":"ContainerStarted","Data":"cc6c2c07675185d7482e28dafef86a2787d8c8d33f31f07b55724d4d5541eed8"}
Jul 30 17:27:24.477854 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.477786    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:24.477854 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.477803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" event={"ID":"d9b0103e-aed1-4ce0-8c7c-234f7563100f","Type":"ContainerStarted","Data":"d9d3b089628d9ccde40b22b8cb36a116bf9e18030fb5e27d8929f31c46cff506"}
Jul 30 17:27:24.492478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.492426    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" podStartSLOduration=1.492408744 podStartE2EDuration="1.492408744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:24.491942346 +0000 UTC m=+3073.749403769" watchObservedRunningTime="2026-07-30 17:27:24.492408744 +0000 UTC m=+3073.749870166"
Jul 30 17:27:24.507965 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:24.507917    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" podStartSLOduration=1.5079013909999999 podStartE2EDuration="1.507901391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:24.507730758 +0000 UTC m=+3073.765192179" watchObservedRunningTime="2026-07-30 17:27:24.507901391 +0000 UTC m=+3073.765362794"
Jul 30 17:27:29.028288 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:29.028250    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:27:35.486183 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:35.486145    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:27:35.486679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:35.486219    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:27:54.041148 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.041106    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:27:54.049160 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.049134    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.051830 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.051808    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"default-dockercfg-xl94t\""
Jul 30 17:27:54.052628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.052608    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"kube-root-ca.crt\""
Jul 30 17:27:54.052725 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.052665    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"openshift-service-ca.crt\""
Jul 30 17:27:54.053226 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.053208    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"auth-test-model-f45123-kserve-self-signed-certs\""
Jul 30 17:27:54.058024 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.058004    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:27:54.065526 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065499    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.065621 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065565    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.065621 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065592    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.065699 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065643    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.065699 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065685    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.065897 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.065711    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vljx\" (UniqueName: \"kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166425 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166390    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166441    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166512    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166602    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vljx\" (UniqueName: \"kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166912 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.166912 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.166891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.167083 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.167019    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.168861 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.168841    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.169026 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.169007    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.180526 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.180502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vljx\" (UniqueName: \"kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx\") pod \"auth-test-model-f45123-kserve-796d7974c5-t8j2r\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") " pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.359504 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.359463    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:27:54.506042 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.506015    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:27:54.508543 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:27:54.508515    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode86dca1d_19d5_494a_a783_1b6ca2fd6844.slice/crio-a151d8ecf91ed57c2546184bf22d7dc84020f44a5ba91ab0b803096e0543a7cb WatchSource:0}: Error finding container a151d8ecf91ed57c2546184bf22d7dc84020f44a5ba91ab0b803096e0543a7cb: Status 404 returned error can't find the container with id a151d8ecf91ed57c2546184bf22d7dc84020f44a5ba91ab0b803096e0543a7cb
Jul 30 17:27:54.632767 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.632672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" event={"ID":"e86dca1d-19d5-494a-a783-1b6ca2fd6844","Type":"ContainerStarted","Data":"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"}
Jul 30 17:27:54.632767 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.632720    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" event={"ID":"e86dca1d-19d5-494a-a783-1b6ca2fd6844","Type":"ContainerStarted","Data":"a151d8ecf91ed57c2546184bf22d7dc84020f44a5ba91ab0b803096e0543a7cb"}
Jul 30 17:27:54.652568 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:54.652476    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" podStartSLOduration=0.652459978 podStartE2EDuration="652.459978ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:54.651265698 +0000 UTC m=+3103.908727117" watchObservedRunningTime="2026-07-30 17:27:54.652459978 +0000 UTC m=+3103.909921397"
Jul 30 17:27:57.527456 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:27:57.527419    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:28:04.360663 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:04.360624    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:28:04.361153 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:04.360686    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:28:04.373594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:04.373572    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:28:04.693091 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:04.693002    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:28:27.430732 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.430693    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:28:27.434445 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.434424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.438085 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.438064    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"openshift-service-ca.crt\""
Jul 30 17:28:27.438213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.438121    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"auth-test-model-27deb9-kserve-self-signed-certs\""
Jul 30 17:28:27.438213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.438145    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"default-dockercfg-lgkhh\""
Jul 30 17:28:27.438213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.438122    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"kube-root-ca.crt\""
Jul 30 17:28:27.446355 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.446332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:28:27.540161 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.540345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540183    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.540345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540205    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.540345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540222    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.540345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97glb\" (UniqueName: \"kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.540345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.540264    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.640995 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.640963    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.640995 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.640995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641123    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97glb\" (UniqueName: \"kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641213 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641379 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641236    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641379 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641317    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641479 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641397    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.641586 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.641568    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.643340 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.643322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.643570 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.643552    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.649307 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.649281    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97glb\" (UniqueName: \"kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb\") pod \"auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") " pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.744151 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.744081    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:27.883383 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:27.883343    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:28:27.886028 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:28:27.886001    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f50f527_e3f4_4f84_91fc_48c68da6b042.slice/crio-20670ac6376a43d5e64dde40c448f0b16abfc383d1c91d63baa263476261767f WatchSource:0}: Error finding container 20670ac6376a43d5e64dde40c448f0b16abfc383d1c91d63baa263476261767f: Status 404 returned error can't find the container with id 20670ac6376a43d5e64dde40c448f0b16abfc383d1c91d63baa263476261767f
Jul 30 17:28:28.800614 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:28.800578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" event={"ID":"5f50f527-e3f4-4f84-91fc-48c68da6b042","Type":"ContainerStarted","Data":"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"}
Jul 30 17:28:28.800614 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:28.800621    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" event={"ID":"5f50f527-e3f4-4f84-91fc-48c68da6b042","Type":"ContainerStarted","Data":"20670ac6376a43d5e64dde40c448f0b16abfc383d1c91d63baa263476261767f"}
Jul 30 17:28:28.835766 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:28.835716    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" podStartSLOduration=1.835700511 podStartE2EDuration="1.835700511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:28:28.834968376 +0000 UTC m=+3138.092429799" watchObservedRunningTime="2026-07-30 17:28:28.835700511 +0000 UTC m=+3138.093161927"
Jul 30 17:28:31.537517 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:31.537469    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:28:37.745195 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:37.745110    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:37.745195 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:37.745164    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:37.758051 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:37.758026    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:28:37.856104 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:28:37.856072    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:29:03.620263 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:29:03.620226    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:29:29.430715 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:29:29.430681    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:29:47.327109 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:29:47.327075    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:30:08.015119 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:30:08.015074    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:30:34.633886 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:30:34.633845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:30:50.331609 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:30:50.331570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:31:11.433012 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.432987    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:31:11.436169 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.436144    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:31:11.438442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.438423    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:31:11.442009 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.441992    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:31:11.625990 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.625920    2561 scope.go:117] "RemoveContainer" containerID="8a3eebcefe3169b9e88502ad9cf70a94096bd267dc7ffd9bd8764a1a6c4e4f4a"
Jul 30 17:31:11.634347 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:11.634330    2561 scope.go:117] "RemoveContainer" containerID="c60546d5d3791e257c1f42f9f333fded71f44d8ef6069d3c378e8499f21639e6"
Jul 30 17:31:16.319382 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:16.319340    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:31:33.524002 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:33.523924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:31:58.919861 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:31:58.919823    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:32:11.656121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:32:11.656015    2561 scope.go:117] "RemoveContainer" containerID="7c8ea8f96e358433f2e18446190db17d20addbdf73e127e52ecfac88f6da5b62"
Jul 30 17:32:17.039306 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:32:17.039257    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:32:44.318384 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:32:44.318350    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:33:02.221651 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:02.221615    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:33:11.675900 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:11.675867    2561 scope.go:117] "RemoveContainer" containerID="7b86cfae37c45e797d8c89e7e23c360fa2c2fa3d4b15b0482ebf0b0b2a6db819"
Jul 30 17:33:28.436335 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:28.436303    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:33:33.882927 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:33.882887    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:33:34.729073 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:34.729033    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:33:39.009991 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.009956    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:33:39.012345 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.012328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.014982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.014960    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"sub-test-model-f45123-kserve-self-signed-certs\""
Jul 30 17:33:39.025577 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.025545    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:33:39.102831 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102799    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.102831 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.103020 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102860    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.103020 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102888    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.103020 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hbw2\" (UniqueName: \"kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.103020 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.102973    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204352 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204352 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204352    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204395    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204414    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hbw2\" (UniqueName: \"kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204836 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204855    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.204938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.204881    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.206655 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.206635    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.206824 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.206806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.215436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.215416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hbw2\" (UniqueName: \"kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2\") pod \"sub-test-model-f45123-kserve-8667f8fc69-jhkzx\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") " pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.321458 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.321392    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:39.472135 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:39.472100    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:33:39.475081 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:33:39.475053    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60495c57_342d_4aa8_818e_f9eac017df5d.slice/crio-19e0de026c9c7d11d7928fb1b1595c5469ff3e3a65c1cd3bc454dab0facf1546 WatchSource:0}: Error finding container 19e0de026c9c7d11d7928fb1b1595c5469ff3e3a65c1cd3bc454dab0facf1546: Status 404 returned error can't find the container with id 19e0de026c9c7d11d7928fb1b1595c5469ff3e3a65c1cd3bc454dab0facf1546
Jul 30 17:33:40.384922 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:40.384883    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" event={"ID":"60495c57-342d-4aa8-818e-f9eac017df5d","Type":"ContainerStarted","Data":"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"}
Jul 30 17:33:40.385304 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:40.384928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" event={"ID":"60495c57-342d-4aa8-818e-f9eac017df5d","Type":"ContainerStarted","Data":"19e0de026c9c7d11d7928fb1b1595c5469ff3e3a65c1cd3bc454dab0facf1546"}
Jul 30 17:33:40.406152 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:40.406068    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" podStartSLOduration=2.406054616 podStartE2EDuration="2.406054616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:40.404840513 +0000 UTC m=+3449.662301935" watchObservedRunningTime="2026-07-30 17:33:40.406054616 +0000 UTC m=+3449.663516106"
Jul 30 17:33:45.630327 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:45.630294    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:33:48.901884 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:48.901815    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" podUID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" containerName="main" containerID="cri-o://aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0" gracePeriod=45
Jul 30 17:33:49.161101 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.161043    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:33:49.197534 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197506    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197678 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197567    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197678 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197631    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197678 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197659    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vljx\" (UniqueName: \"kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197842 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197686    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197842 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197726    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache\") pod \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\" (UID: \"e86dca1d-19d5-494a-a783-1b6ca2fd6844\") "
Jul 30 17:33:49.197842 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197826    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home" (OuterVolumeSpecName: "home") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:49.197988 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.197936    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:49.198075 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.198056    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache" (OuterVolumeSpecName: "model-cache") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:49.198131 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.198116    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.198186 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.198139    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.199859 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.199830    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm" (OuterVolumeSpecName: "dshm") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:49.199859 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.199840    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:49.200004 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.199889    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx" (OuterVolumeSpecName: "kube-api-access-5vljx") pod "e86dca1d-19d5-494a-a783-1b6ca2fd6844" (UID: "e86dca1d-19d5-494a-a783-1b6ca2fd6844"). InnerVolumeSpecName "kube-api-access-5vljx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:33:49.298744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.298714    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vljx\" (UniqueName: \"kubernetes.io/projected/e86dca1d-19d5-494a-a783-1b6ca2fd6844-kube-api-access-5vljx\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.298744 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.298739    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86dca1d-19d5-494a-a783-1b6ca2fd6844-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.298911 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.298751    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.298911 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.298762    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e86dca1d-19d5-494a-a783-1b6ca2fd6844-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:49.321538 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.321513    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:49.321641 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.321551    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:49.334061 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.334041    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:49.428451 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.428384    2561 generic.go:356] "Generic (PLEG): container finished" podID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" containerID="aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0" exitCode=0
Jul 30 17:33:49.428451 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.428424    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"
Jul 30 17:33:49.428623 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.428469    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" event={"ID":"e86dca1d-19d5-494a-a783-1b6ca2fd6844","Type":"ContainerDied","Data":"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"}
Jul 30 17:33:49.428623 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.428526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r" event={"ID":"e86dca1d-19d5-494a-a783-1b6ca2fd6844","Type":"ContainerDied","Data":"a151d8ecf91ed57c2546184bf22d7dc84020f44a5ba91ab0b803096e0543a7cb"}
Jul 30 17:33:49.428623 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.428551    2561 scope.go:117] "RemoveContainer" containerID="aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"
Jul 30 17:33:49.437632 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.437616    2561 scope.go:117] "RemoveContainer" containerID="aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"
Jul 30 17:33:49.437899 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:33:49.437870    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0\": container with ID starting with aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0 not found: ID does not exist" containerID="aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"
Jul 30 17:33:49.437985 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.437898    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0"} err="failed to get container status \"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0\": rpc error: code = NotFound desc = could not find container \"aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0\": container with ID starting with aa075cc1e2cd7fcf4a00aeeeb68cb54b79f5076a2bb9008449af9f4cbfd6a4d0 not found: ID does not exist"
Jul 30 17:33:49.440752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.440733    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:33:49.446427 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.446408    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:33:49.453660 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.453641    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/auth-test-model-f45123-kserve-796d7974c5-t8j2r"]
Jul 30 17:33:49.748004 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:49.747908    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" podUID="5f50f527-e3f4-4f84-91fc-48c68da6b042" containerName="main" containerID="cri-o://ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8" gracePeriod=45
Jul 30 17:33:50.009551 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.007798    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:33:50.104823 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.104791    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.104982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.104852    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.104982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.104885    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.104982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.104940    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.104982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.104982    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.105214 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.105026    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97glb\" (UniqueName: \"kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb\") pod \"5f50f527-e3f4-4f84-91fc-48c68da6b042\" (UID: \"5f50f527-e3f4-4f84-91fc-48c68da6b042\") "
Jul 30 17:33:50.105214 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.105063    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:50.105317 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.105234    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.105317 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.105241    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:50.105317 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.105253    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home" (OuterVolumeSpecName: "home") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:50.107092 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.107062    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm" (OuterVolumeSpecName: "dshm") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:50.107092 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.107072    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:50.107262 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.107109    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb" (OuterVolumeSpecName: "kube-api-access-97glb") pod "5f50f527-e3f4-4f84-91fc-48c68da6b042" (UID: "5f50f527-e3f4-4f84-91fc-48c68da6b042"). InnerVolumeSpecName "kube-api-access-97glb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:33:50.205938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.205901    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.205938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.205925    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.205938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.205934    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97glb\" (UniqueName: \"kubernetes.io/projected/5f50f527-e3f4-4f84-91fc-48c68da6b042-kube-api-access-97glb\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.205938 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.205945    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f50f527-e3f4-4f84-91fc-48c68da6b042-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.206206 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.205953    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f50f527-e3f4-4f84-91fc-48c68da6b042-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:33:50.436151 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.436121    2561 generic.go:356] "Generic (PLEG): container finished" podID="5f50f527-e3f4-4f84-91fc-48c68da6b042" containerID="ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8" exitCode=0
Jul 30 17:33:50.436346 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.436162    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"
Jul 30 17:33:50.436346 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.436193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" event={"ID":"5f50f527-e3f4-4f84-91fc-48c68da6b042","Type":"ContainerDied","Data":"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"}
Jul 30 17:33:50.436346 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.436241    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m" event={"ID":"5f50f527-e3f4-4f84-91fc-48c68da6b042","Type":"ContainerDied","Data":"20670ac6376a43d5e64dde40c448f0b16abfc383d1c91d63baa263476261767f"}
Jul 30 17:33:50.436346 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.436263    2561 scope.go:117] "RemoveContainer" containerID="ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"
Jul 30 17:33:50.448983 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.448969    2561 scope.go:117] "RemoveContainer" containerID="ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"
Jul 30 17:33:50.449244 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:33:50.449227    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8\": container with ID starting with ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8 not found: ID does not exist" containerID="ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"
Jul 30 17:33:50.449328 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.449250    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8"} err="failed to get container status \"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8\": rpc error: code = NotFound desc = could not find container \"ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8\": container with ID starting with ece535d5dfaf5fd21c9cd498e7e2f0f75ed1b2dff229967ae2d998e6e8166ab8 not found: ID does not exist"
Jul 30 17:33:50.462286 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.462261    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:33:50.467771 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:50.467747    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/auth-test-model-27deb9-kserve-6bd5c6d9c9-6cp8m"]
Jul 30 17:33:51.342390 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:51.342347    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f50f527-e3f4-4f84-91fc-48c68da6b042" path="/var/lib/kubelet/pods/5f50f527-e3f4-4f84-91fc-48c68da6b042/volumes"
Jul 30 17:33:51.342931 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:33:51.342909    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" path="/var/lib/kubelet/pods/e86dca1d-19d5-494a-a783-1b6ca2fd6844/volumes"
Jul 30 17:34:13.631407 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631374    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:34:13.631835 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631762    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f50f527-e3f4-4f84-91fc-48c68da6b042" containerName="main"
Jul 30 17:34:13.631835 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631775    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f50f527-e3f4-4f84-91fc-48c68da6b042" containerName="main"
Jul 30 17:34:13.631835 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631789    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" containerName="main"
Jul 30 17:34:13.631835 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631795    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" containerName="main"
Jul 30 17:34:13.631994 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631851    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f50f527-e3f4-4f84-91fc-48c68da6b042" containerName="main"
Jul 30 17:34:13.631994 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.631861    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86dca1d-19d5-494a-a783-1b6ca2fd6844" containerName="main"
Jul 30 17:34:13.636114 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.636099    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.639768 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.639732    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"openshift-service-ca.crt\""
Jul 30 17:34:13.639768 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.639762    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"sub-test-model-27deb9-kserve-self-signed-certs\""
Jul 30 17:34:13.639981 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.639778    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"default-dockercfg-lgkhh\""
Jul 30 17:34:13.639981 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.639793    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"kube-root-ca.crt\""
Jul 30 17:34:13.647339 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.647269    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:34:13.808276 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.808478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.808478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prdtx\" (UniqueName: \"kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.808478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808384    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.808478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.808478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.808431    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909143 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909048    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909143 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909089    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909143 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prdtx\" (UniqueName: \"kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909429 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909429 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909190    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909429 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909219    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909636 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909525    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909636 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909570    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.909636 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.909590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.911473 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.911449    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.911685 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.911668    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.918559 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.918540    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prdtx\" (UniqueName: \"kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx\") pod \"sub-test-model-27deb9-kserve-6fd59864b9-trf5x\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") " pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:13.946545 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:13.946520    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:14.082606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:14.082580    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:34:14.084871 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:34:14.084831    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0a0b0b9_1964_4e29_a93e_6d40736d17a0.slice/crio-a7104cce6ea8dc2bafaeea24ec8dbe9ac9b3252700ec4da2b5898ede07403cef WatchSource:0}: Error finding container a7104cce6ea8dc2bafaeea24ec8dbe9ac9b3252700ec4da2b5898ede07403cef: Status 404 returned error can't find the container with id a7104cce6ea8dc2bafaeea24ec8dbe9ac9b3252700ec4da2b5898ede07403cef
Jul 30 17:34:14.555606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:14.555569    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" event={"ID":"e0a0b0b9-1964-4e29-a93e-6d40736d17a0","Type":"ContainerStarted","Data":"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"}
Jul 30 17:34:14.555606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:14.555611    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" event={"ID":"e0a0b0b9-1964-4e29-a93e-6d40736d17a0","Type":"ContainerStarted","Data":"a7104cce6ea8dc2bafaeea24ec8dbe9ac9b3252700ec4da2b5898ede07403cef"}
Jul 30 17:34:14.574465 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:14.574406    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" podStartSLOduration=1.574387046 podStartE2EDuration="1.574387046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:34:14.572896326 +0000 UTC m=+3483.830357756" watchObservedRunningTime="2026-07-30 17:34:14.574387046 +0000 UTC m=+3483.831848466"
Jul 30 17:34:17.132576 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:17.132535    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:34:23.947005 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:23.946972    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:23.947508 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:23.947043    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:23.959748 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:23.959722    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:34:24.617360 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:34:24.617331    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:35:03.523764 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:35:03.523732    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:35:20.326475 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:35:20.326441    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:35:45.118667 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:35:45.118629    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:35:50.306547 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:35:50.306514    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:35:51.357607 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:35:51.357574    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:36:03.620589 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:03.620502    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:36:04.836204 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.836175    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:36:04.839551 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.839530    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.842223 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.842204    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f45123\"/\"rate-test-model-f45123-kserve-self-signed-certs\""
Jul 30 17:36:04.850977 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.850953    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:36:04.950321 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950278    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.950321 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.950552 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950347    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.950552 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950391    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.950552 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950427    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjhh\" (UniqueName: \"kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:04.950552 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:04.950538    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051702 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051738    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051766    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051832    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.051863 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.051854    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjhh\" (UniqueName: \"kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.052140 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.052123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.052199 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.052160    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.052247 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.052229    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.054041 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.054021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.054292 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.054273    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.059442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.059419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjhh\" (UniqueName: \"kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh\") pod \"rate-test-model-f45123-kserve-5697bb59dc-xlmwf\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") " pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.148734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.148648    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:05.288177 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.288149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:36:05.291013 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:36:05.290987    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83bc52a3_d19a_4c58_9f7e_55d8fe312ba7.slice/crio-c83623fc63716a016886328341d16a74b15e4abed93c386b229e0d56462dd546 WatchSource:0}: Error finding container c83623fc63716a016886328341d16a74b15e4abed93c386b229e0d56462dd546: Status 404 returned error can't find the container with id c83623fc63716a016886328341d16a74b15e4abed93c386b229e0d56462dd546
Jul 30 17:36:05.325811 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.325784    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" podUID="60495c57-342d-4aa8-818e-f9eac017df5d" containerName="main" containerID="cri-o://f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475" gracePeriod=45
Jul 30 17:36:05.581115 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.581091    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:36:05.657967 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.657937    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658127 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.657997    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658127 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658020    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658127 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658060    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hbw2\" (UniqueName: \"kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658127 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658117    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658159    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs\") pod \"60495c57-342d-4aa8-818e-f9eac017df5d\" (UID: \"60495c57-342d-4aa8-818e-f9eac017df5d\") "
Jul 30 17:36:05.658359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658313    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home" (OuterVolumeSpecName: "home") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:05.658359 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658323    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:05.658530 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658419    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:05.658530 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658459    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:05.658530 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.658473    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:05.660305 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.660277    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm" (OuterVolumeSpecName: "dshm") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:05.660448 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.660337    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2" (OuterVolumeSpecName: "kube-api-access-5hbw2") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "kube-api-access-5hbw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:36:05.660685 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.660660    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60495c57-342d-4aa8-818e-f9eac017df5d" (UID: "60495c57-342d-4aa8-818e-f9eac017df5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:36:05.759752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.759674    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:05.759752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.759705    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hbw2\" (UniqueName: \"kubernetes.io/projected/60495c57-342d-4aa8-818e-f9eac017df5d-kube-api-access-5hbw2\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:05.759752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.759720    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60495c57-342d-4aa8-818e-f9eac017df5d-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:05.759752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:05.759734    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60495c57-342d-4aa8-818e-f9eac017df5d-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.100460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.100427    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" event={"ID":"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7","Type":"ContainerStarted","Data":"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"}
Jul 30 17:36:06.100876 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.100469    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" event={"ID":"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7","Type":"ContainerStarted","Data":"c83623fc63716a016886328341d16a74b15e4abed93c386b229e0d56462dd546"}
Jul 30 17:36:06.101743 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.101719    2561 generic.go:356] "Generic (PLEG): container finished" podID="60495c57-342d-4aa8-818e-f9eac017df5d" containerID="f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475" exitCode=0
Jul 30 17:36:06.101883 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.101754    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"
Jul 30 17:36:06.101883 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.101766    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" event={"ID":"60495c57-342d-4aa8-818e-f9eac017df5d","Type":"ContainerDied","Data":"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"}
Jul 30 17:36:06.101883 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.101800    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx" event={"ID":"60495c57-342d-4aa8-818e-f9eac017df5d","Type":"ContainerDied","Data":"19e0de026c9c7d11d7928fb1b1595c5469ff3e3a65c1cd3bc454dab0facf1546"}
Jul 30 17:36:06.101883 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.101823    2561 scope.go:117] "RemoveContainer" containerID="f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"
Jul 30 17:36:06.111442 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.111424    2561 scope.go:117] "RemoveContainer" containerID="f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"
Jul 30 17:36:06.111702 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:36:06.111682    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475\": container with ID starting with f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475 not found: ID does not exist" containerID="f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"
Jul 30 17:36:06.111773 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.111714    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475"} err="failed to get container status \"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475\": rpc error: code = NotFound desc = could not find container \"f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475\": container with ID starting with f58afdca2873811103be5b6419e637cfd8a0d79226efad013d1ed747cfa6f475 not found: ID does not exist"
Jul 30 17:36:06.117903 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.117863    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" podStartSLOduration=2.117847632 podStartE2EDuration="2.117847632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:06.117804561 +0000 UTC m=+3595.375265983" watchObservedRunningTime="2026-07-30 17:36:06.117847632 +0000 UTC m=+3595.375309055"
Jul 30 17:36:06.134673 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.134649    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:36:06.138044 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.138023    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/sub-test-model-f45123-kserve-8667f8fc69-jhkzx"]
Jul 30 17:36:06.376427 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.376383    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" podUID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" containerName="main" containerID="cri-o://8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935" gracePeriod=45
Jul 30 17:36:06.620515 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.620474    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:36:06.767044 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.766955    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767044 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767042    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767067    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767127    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767178    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prdtx\" (UniqueName: \"kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767207    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm\") pod \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\" (UID: \"e0a0b0b9-1964-4e29-a93e-6d40736d17a0\") "
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767224    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:06.767269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767255    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:06.767619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767311    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home" (OuterVolumeSpecName: "home") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:06.767619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767540    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.767619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767574    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.767619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.767587    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.769314 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.769287    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm" (OuterVolumeSpecName: "dshm") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:36:06.769516 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.769466    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx" (OuterVolumeSpecName: "kube-api-access-prdtx") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "kube-api-access-prdtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:36:06.769759 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.769741    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0a0b0b9-1964-4e29-a93e-6d40736d17a0" (UID: "e0a0b0b9-1964-4e29-a93e-6d40736d17a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:36:06.868096 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.868069    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.868096 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.868092    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prdtx\" (UniqueName: \"kubernetes.io/projected/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-kube-api-access-prdtx\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:06.868096 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:06.868101    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a0b0b9-1964-4e29-a93e-6d40736d17a0-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:36:07.108152 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.108121    2561 generic.go:356] "Generic (PLEG): container finished" podID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" containerID="8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935" exitCode=0
Jul 30 17:36:07.108609 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.108162    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"
Jul 30 17:36:07.108609 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.108207    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" event={"ID":"e0a0b0b9-1964-4e29-a93e-6d40736d17a0","Type":"ContainerDied","Data":"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"}
Jul 30 17:36:07.108609 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.108251    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x" event={"ID":"e0a0b0b9-1964-4e29-a93e-6d40736d17a0","Type":"ContainerDied","Data":"a7104cce6ea8dc2bafaeea24ec8dbe9ac9b3252700ec4da2b5898ede07403cef"}
Jul 30 17:36:07.108609 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.108275    2561 scope.go:117] "RemoveContainer" containerID="8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"
Jul 30 17:36:07.117679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.117662    2561 scope.go:117] "RemoveContainer" containerID="8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"
Jul 30 17:36:07.117934 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:36:07.117916    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935\": container with ID starting with 8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935 not found: ID does not exist" containerID="8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"
Jul 30 17:36:07.117974 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.117941    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935"} err="failed to get container status \"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935\": rpc error: code = NotFound desc = could not find container \"8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935\": container with ID starting with 8ad6cebc58895ee9beadf3922518483d43e4bfee3d9ed81b6d84deab84766935 not found: ID does not exist"
Jul 30 17:36:07.129457 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.129435    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:36:07.133676 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.133656    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/sub-test-model-27deb9-kserve-6fd59864b9-trf5x"]
Jul 30 17:36:07.341373 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.341338    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60495c57-342d-4aa8-818e-f9eac017df5d" path="/var/lib/kubelet/pods/60495c57-342d-4aa8-818e-f9eac017df5d/volumes"
Jul 30 17:36:07.341762 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:07.341748    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" path="/var/lib/kubelet/pods/e0a0b0b9-1964-4e29-a93e-6d40736d17a0/volumes"
Jul 30 17:36:11.459448 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:11.459422    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:36:11.464370 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:11.464350    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:36:11.465801 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:11.465780    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:36:11.470786 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:11.470772    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:36:13.519101 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:13.519071    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:36:15.149283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:15.149248    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:15.149283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:15.149293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:15.162460 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:15.162437    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:16.172444 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:16.172411    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:36:25.430001 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.429969    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:36:25.430548 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430530    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" containerName="main"
Jul 30 17:36:25.430648 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430551    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" containerName="main"
Jul 30 17:36:25.430648 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430568    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60495c57-342d-4aa8-818e-f9eac017df5d" containerName="main"
Jul 30 17:36:25.430648 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430577    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="60495c57-342d-4aa8-818e-f9eac017df5d" containerName="main"
Jul 30 17:36:25.430811 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430675    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="60495c57-342d-4aa8-818e-f9eac017df5d" containerName="main"
Jul 30 17:36:25.430811 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.430690    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0a0b0b9-1964-4e29-a93e-6d40736d17a0" containerName="main"
Jul 30 17:36:25.433693 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.433675    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.436129 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.436112    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"default-dockercfg-lgkhh\""
Jul 30 17:36:25.437053 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.437036    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"rate-test-model-27deb9-kserve-self-signed-certs\""
Jul 30 17:36:25.437124 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.437076    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"kube-root-ca.crt\""
Jul 30 17:36:25.437173 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.437073    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27deb9\"/\"openshift-service-ca.crt\""
Jul 30 17:36:25.442470 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.442449    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:36:25.527911 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.527873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.528110 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.527924    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.528110 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.528016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.528210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.528105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t59v\" (UniqueName: \"kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.528210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.528138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.528210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.528157    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.628860 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.628826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629022 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.628903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t59v\" (UniqueName: \"kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629022 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.628933    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629022 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.628956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629189 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.629085    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629189 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.629172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629339 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.629321    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629546 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.629528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.629546 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.629536    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.631730 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.631706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.631947 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.631927    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.638955 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.638934    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t59v\" (UniqueName: \"kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v\") pod \"rate-test-model-27deb9-kserve-59c75c7d7-t9f46\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") " pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.744190 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.744122    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:25.885343 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:25.885301    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:36:25.887391 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:36:25.887367    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podece299e0_879f_455d_af4f_ffa0f11df858.slice/crio-f931dfedc7d559d44e600af353fece83fa8a32089035a174338797dc71067d60 WatchSource:0}: Error finding container f931dfedc7d559d44e600af353fece83fa8a32089035a174338797dc71067d60: Status 404 returned error can't find the container with id f931dfedc7d559d44e600af353fece83fa8a32089035a174338797dc71067d60
Jul 30 17:36:26.211060 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:26.211025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" event={"ID":"ece299e0-879f-455d-af4f-ffa0f11df858","Type":"ContainerStarted","Data":"66210f56652416d675b410d50514968245095f0cef3d261854de373a41fd7154"}
Jul 30 17:36:26.211060 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:26.211064    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" event={"ID":"ece299e0-879f-455d-af4f-ffa0f11df858","Type":"ContainerStarted","Data":"f931dfedc7d559d44e600af353fece83fa8a32089035a174338797dc71067d60"}
Jul 30 17:36:26.234582 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:26.234533    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" podStartSLOduration=1.2345175290000001 podStartE2EDuration="1.234517529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:26.23330213 +0000 UTC m=+3615.490763550" watchObservedRunningTime="2026-07-30 17:36:26.234517529 +0000 UTC m=+3615.491978948"
Jul 30 17:36:28.823244 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:28.823210    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:36:35.744677 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:35.744644    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:35.745136 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:35.744714    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:35.757175 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:35.757147    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:36:36.274195 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:36:36.274167    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:37:25.236237 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:37:25.236201    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:37:46.021543 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:37:46.021449    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:38:08.017951 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:08.017916    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:38:13.873053 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:13.873019    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:38:14.612673 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:14.612642    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:38:18.831523 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:18.831474    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135639216"]
Jul 30 17:38:18.834996 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:18.834979    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:18.837419 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:18.837402    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4sss8\""
Jul 30 17:38:18.841809 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:18.841779    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135639216"]
Jul 30 17:38:18.905836 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:18.905805    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfnn\" (UniqueName: \"kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn\") pod \"test-curl-7785-140008135639216\" (UID: \"e1b4261d-53f5-48b3-b4f6-be97cbec4903\") " pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:19.006316 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.006284    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfnn\" (UniqueName: \"kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn\") pod \"test-curl-7785-140008135639216\" (UID: \"e1b4261d-53f5-48b3-b4f6-be97cbec4903\") " pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:19.014886 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.014858    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfnn\" (UniqueName: \"kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn\") pod \"test-curl-7785-140008135639216\" (UID: \"e1b4261d-53f5-48b3-b4f6-be97cbec4903\") " pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:19.144684 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.144607    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:19.284735 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.284699    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135639216"]
Jul 30 17:38:19.295863 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:19.295837    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1b4261d_53f5_48b3_b4f6_be97cbec4903.slice/crio-9da7cfb603ef9247804e74116e2db47698de7342851cece41ab9b84122cb8f22 WatchSource:0}: Error finding container 9da7cfb603ef9247804e74116e2db47698de7342851cece41ab9b84122cb8f22: Status 404 returned error can't find the container with id 9da7cfb603ef9247804e74116e2db47698de7342851cece41ab9b84122cb8f22
Jul 30 17:38:19.297555 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.297537    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:38:19.770770 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:19.770732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135639216" event={"ID":"e1b4261d-53f5-48b3-b4f6-be97cbec4903","Type":"ContainerStarted","Data":"9da7cfb603ef9247804e74116e2db47698de7342851cece41ab9b84122cb8f22"}
Jul 30 17:38:20.776994 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:20.776959    2561 generic.go:356] "Generic (PLEG): container finished" podID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" containerID="77f4d73bfcc0a3d3f5937945ad1c4d0f606c913425da851e9e558346447879b2" exitCode=0
Jul 30 17:38:20.777375 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:20.777049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135639216" event={"ID":"e1b4261d-53f5-48b3-b4f6-be97cbec4903","Type":"ContainerDied","Data":"77f4d73bfcc0a3d3f5937945ad1c4d0f606c913425da851e9e558346447879b2"}
Jul 30 17:38:21.126879 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:21.126845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:38:21.923350 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:21.923325    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:21.924891 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:21.924869    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntfnn\" (UniqueName: \"kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn\") pod \"e1b4261d-53f5-48b3-b4f6-be97cbec4903\" (UID: \"e1b4261d-53f5-48b3-b4f6-be97cbec4903\") "
Jul 30 17:38:21.926862 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:21.926836    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn" (OuterVolumeSpecName: "kube-api-access-ntfnn") pod "e1b4261d-53f5-48b3-b4f6-be97cbec4903" (UID: "e1b4261d-53f5-48b3-b4f6-be97cbec4903"). InnerVolumeSpecName "kube-api-access-ntfnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:21.974987 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:21.974958    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135639216_e1b4261d-53f5-48b3-b4f6-be97cbec4903/test-curl-7785-140008135639216/0.log"
Jul 30 17:38:22.002433 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.002405    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135639216"]
Jul 30 17:38:22.006663 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.006638    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135639216"]
Jul 30 17:38:22.026349 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.026324    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntfnn\" (UniqueName: \"kubernetes.io/projected/e1b4261d-53f5-48b3-b4f6-be97cbec4903-kube-api-access-ntfnn\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:22.173262 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.173234    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:22.173669 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.173654    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" containerName="test-curl-7785-140008135639216"
Jul 30 17:38:22.173719 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.173673    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" containerName="test-curl-7785-140008135639216"
Jul 30 17:38:22.173769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.173759    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" containerName="test-curl-7785-140008135639216"
Jul 30 17:38:22.176789 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.176770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:22.185353 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.185334    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:22.227581 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.227552    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2fx\" (UniqueName: \"kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx\") pod \"test-curl-7785-140008135781520\" (UID: \"6704dc38-0c2c-46f5-b1e3-c84b2472b352\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:22.328799 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.328763    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2fx\" (UniqueName: \"kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx\") pod \"test-curl-7785-140008135781520\" (UID: \"6704dc38-0c2c-46f5-b1e3-c84b2472b352\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:22.337933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.337908    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2fx\" (UniqueName: \"kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx\") pod \"test-curl-7785-140008135781520\" (UID: \"6704dc38-0c2c-46f5-b1e3-c84b2472b352\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:22.486312 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.486238    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:22.619194 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.619170    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:22.621079 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:22.621053    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6704dc38_0c2c_46f5_b1e3_c84b2472b352.slice/crio-5f80d7abb015ee3e9816528954ef677075d6bd2eca0a33d674166db98fa689a2 WatchSource:0}: Error finding container 5f80d7abb015ee3e9816528954ef677075d6bd2eca0a33d674166db98fa689a2: Status 404 returned error can't find the container with id 5f80d7abb015ee3e9816528954ef677075d6bd2eca0a33d674166db98fa689a2
Jul 30 17:38:22.788248 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.788170    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9da7cfb603ef9247804e74116e2db47698de7342851cece41ab9b84122cb8f22"
Jul 30 17:38:22.788248 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.788180    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:22.789417 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:22.789390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135781520" event={"ID":"6704dc38-0c2c-46f5-b1e3-c84b2472b352","Type":"ContainerStarted","Data":"5f80d7abb015ee3e9816528954ef677075d6bd2eca0a33d674166db98fa689a2"}
Jul 30 17:38:22.798426 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:38:22.798399    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7785-140008135639216\" is forbidden: User \"system:node:ip-10-0-132-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-133.ec2.internal' and this object" podUID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" pod="opendatahub/test-curl-7785-140008135639216"
Jul 30 17:38:23.340947 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:23.340912    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b4261d-53f5-48b3-b4f6-be97cbec4903" path="/var/lib/kubelet/pods/e1b4261d-53f5-48b3-b4f6-be97cbec4903/volumes"
Jul 30 17:38:23.796626 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:23.796537    2561 generic.go:356] "Generic (PLEG): container finished" podID="6704dc38-0c2c-46f5-b1e3-c84b2472b352" containerID="ff00b5ed208c03229c0b8cb7fa27ef709bb87d86010c5e9f34f2f05b89b737df" exitCode=0
Jul 30 17:38:23.796775 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:23.796621    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135781520" event={"ID":"6704dc38-0c2c-46f5-b1e3-c84b2472b352","Type":"ContainerDied","Data":"ff00b5ed208c03229c0b8cb7fa27ef709bb87d86010c5e9f34f2f05b89b737df"}
Jul 30 17:38:24.934842 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.934816    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:24.949113 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.949086    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj2fx\" (UniqueName: \"kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx\") pod \"6704dc38-0c2c-46f5-b1e3-c84b2472b352\" (UID: \"6704dc38-0c2c-46f5-b1e3-c84b2472b352\") "
Jul 30 17:38:24.951469 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.951448    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx" (OuterVolumeSpecName: "kube-api-access-dj2fx") pod "6704dc38-0c2c-46f5-b1e3-c84b2472b352" (UID: "6704dc38-0c2c-46f5-b1e3-c84b2472b352"). InnerVolumeSpecName "kube-api-access-dj2fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:24.955819 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.955795    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135781520_6704dc38-0c2c-46f5-b1e3-c84b2472b352/test-curl-7785-140008135781520/0.log"
Jul 30 17:38:24.982916 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.982892    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:24.988945 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:24.988927    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:25.049811 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:25.049790    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj2fx\" (UniqueName: \"kubernetes.io/projected/6704dc38-0c2c-46f5-b1e3-c84b2472b352-kube-api-access-dj2fx\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:25.340372 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:25.340337    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6704dc38-0c2c-46f5-b1e3-c84b2472b352" path="/var/lib/kubelet/pods/6704dc38-0c2c-46f5-b1e3-c84b2472b352/volumes"
Jul 30 17:38:25.807353 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:25.807265    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.807353 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:25.807300    2561 scope.go:117] "RemoveContainer" containerID="ff00b5ed208c03229c0b8cb7fa27ef709bb87d86010c5e9f34f2f05b89b737df"
Jul 30 17:38:27.859335 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.859299    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:27.859783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.859725    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6704dc38-0c2c-46f5-b1e3-c84b2472b352" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:27.859783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.859739    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6704dc38-0c2c-46f5-b1e3-c84b2472b352" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:27.859901 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.859807    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6704dc38-0c2c-46f5-b1e3-c84b2472b352" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:27.863827 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.863806    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:27.866731 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.866706    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4sss8\""
Jul 30 17:38:27.869820 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.869794    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:27.874158 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.874132    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smm95\" (UniqueName: \"kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95\") pod \"test-curl-7785-140008135606320\" (UID: \"52a41276-3e4b-4199-ab64-387a2bc6a61d\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:27.974917 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.974884    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smm95\" (UniqueName: \"kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95\") pod \"test-curl-7785-140008135606320\" (UID: \"52a41276-3e4b-4199-ab64-387a2bc6a61d\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:27.983471 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:27.983436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smm95\" (UniqueName: \"kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95\") pod \"test-curl-7785-140008135606320\" (UID: \"52a41276-3e4b-4199-ab64-387a2bc6a61d\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:28.175255 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.175175    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:28.310078 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.310000    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:28.312713 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:28.312688    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52a41276_3e4b_4199_ab64_387a2bc6a61d.slice/crio-e36daf9eb3a50a989a1214b14fbaea818573eef821b71c88a7b94e7caec29e35 WatchSource:0}: Error finding container e36daf9eb3a50a989a1214b14fbaea818573eef821b71c88a7b94e7caec29e35: Status 404 returned error can't find the container with id e36daf9eb3a50a989a1214b14fbaea818573eef821b71c88a7b94e7caec29e35
Jul 30 17:38:28.826681 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.826647    2561 generic.go:356] "Generic (PLEG): container finished" podID="52a41276-3e4b-4199-ab64-387a2bc6a61d" containerID="27ff43d1ec05f172c416653983207dd1f617c1b87f0e1a22b0a23adc53fed520" exitCode=0
Jul 30 17:38:28.826849 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.826730    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135606320" event={"ID":"52a41276-3e4b-4199-ab64-387a2bc6a61d","Type":"ContainerDied","Data":"27ff43d1ec05f172c416653983207dd1f617c1b87f0e1a22b0a23adc53fed520"}
Jul 30 17:38:28.826849 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.826771    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135606320" event={"ID":"52a41276-3e4b-4199-ab64-387a2bc6a61d","Type":"ContainerStarted","Data":"e36daf9eb3a50a989a1214b14fbaea818573eef821b71c88a7b94e7caec29e35"}
Jul 30 17:38:28.892125 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:28.892038    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" podUID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" containerName="main" containerID="cri-o://789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb" gracePeriod=45
Jul 30 17:38:29.144010 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.143950    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:38:29.185491 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjjhh\" (UniqueName: \"kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.185689 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185514    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.185689 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185559    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.185689 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185590    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.185689 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185626    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.186319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185706    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home\") pod \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\" (UID: \"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7\") "
Jul 30 17:38:29.186319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.185891    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.186319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.186046    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.186319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.186066    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache" (OuterVolumeSpecName: "model-cache") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.186319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.186247    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home" (OuterVolumeSpecName: "home") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.188190 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.188162    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh" (OuterVolumeSpecName: "kube-api-access-rjjhh") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "kube-api-access-rjjhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:29.188593 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.188568    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm" (OuterVolumeSpecName: "dshm") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.188702 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.188680    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" (UID: "83bc52a3-d19a-4c58-9f7e-55d8fe312ba7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:38:29.286679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.286640    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.286679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.286672    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjjhh\" (UniqueName: \"kubernetes.io/projected/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-kube-api-access-rjjhh\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.286679 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.286686    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.286906 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.286698    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.286906 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.286709    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:29.631411 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.631368    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" podUID="ece299e0-879f-455d-af4f-ffa0f11df858" containerName="main" containerID="cri-o://66210f56652416d675b410d50514968245095f0cef3d261854de373a41fd7154" gracePeriod=45
Jul 30 17:38:29.834495 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.834450    2561 generic.go:356] "Generic (PLEG): container finished" podID="ece299e0-879f-455d-af4f-ffa0f11df858" containerID="66210f56652416d675b410d50514968245095f0cef3d261854de373a41fd7154" exitCode=0
Jul 30 17:38:29.834631 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.834518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" event={"ID":"ece299e0-879f-455d-af4f-ffa0f11df858","Type":"ContainerDied","Data":"66210f56652416d675b410d50514968245095f0cef3d261854de373a41fd7154"}
Jul 30 17:38:29.835982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.835958    2561 generic.go:356] "Generic (PLEG): container finished" podID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" containerID="789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb" exitCode=0
Jul 30 17:38:29.836103 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.836008    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"
Jul 30 17:38:29.836103 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.836042    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" event={"ID":"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7","Type":"ContainerDied","Data":"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"}
Jul 30 17:38:29.836103 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.836075    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf" event={"ID":"83bc52a3-d19a-4c58-9f7e-55d8fe312ba7","Type":"ContainerDied","Data":"c83623fc63716a016886328341d16a74b15e4abed93c386b229e0d56462dd546"}
Jul 30 17:38:29.836103 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.836091    2561 scope.go:117] "RemoveContainer" containerID="789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"
Jul 30 17:38:29.847413 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.847391    2561 scope.go:117] "RemoveContainer" containerID="789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"
Jul 30 17:38:29.847719 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:38:29.847699    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb\": container with ID starting with 789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb not found: ID does not exist" containerID="789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"
Jul 30 17:38:29.847800 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.847731    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb"} err="failed to get container status \"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb\": rpc error: code = NotFound desc = could not find container \"789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb\": container with ID starting with 789fdfca8d589b555594967a5584df4474fc6eb6abe9f788b7d5c51d375984eb not found: ID does not exist"
Jul 30 17:38:29.856164 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.856139    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:38:29.860556 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.860532    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f45123/rate-test-model-f45123-kserve-5697bb59dc-xlmwf"]
Jul 30 17:38:29.884324 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.884271    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:38:29.971469 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.971446    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:29.992110 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t59v\" (UniqueName: \"kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992297 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992137    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smm95\" (UniqueName: \"kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95\") pod \"52a41276-3e4b-4199-ab64-387a2bc6a61d\" (UID: \"52a41276-3e4b-4199-ab64-387a2bc6a61d\") "
Jul 30 17:38:29.992297 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992173    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992297 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992249    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992297 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992282    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992535 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992305    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992535 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992334    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home\") pod \"ece299e0-879f-455d-af4f-ffa0f11df858\" (UID: \"ece299e0-879f-455d-af4f-ffa0f11df858\") "
Jul 30 17:38:29.992962 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992778    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home" (OuterVolumeSpecName: "home") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.992962 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.992815    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.993262 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.993236    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache" (OuterVolumeSpecName: "model-cache") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.993523 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.993376    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135606320_52a41276-3e4b-4199-ab64-387a2bc6a61d/test-curl-7785-140008135606320/0.log"
Jul 30 17:38:29.994961 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.994923    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v" (OuterVolumeSpecName: "kube-api-access-9t59v") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "kube-api-access-9t59v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:29.995335 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.995300    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm" (OuterVolumeSpecName: "dshm") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:29.995438 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.995405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95" (OuterVolumeSpecName: "kube-api-access-smm95") pod "52a41276-3e4b-4199-ab64-387a2bc6a61d" (UID: "52a41276-3e4b-4199-ab64-387a2bc6a61d"). InnerVolumeSpecName "kube-api-access-smm95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:29.996379 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:29.996354    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ece299e0-879f-455d-af4f-ffa0f11df858" (UID: "ece299e0-879f-455d-af4f-ffa0f11df858"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:38:30.020573 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.020539    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:30.026802 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.026777    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:30.093426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093395    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093422    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece299e0-879f-455d-af4f-ffa0f11df858-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093426 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093429    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093437    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093445    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t59v\" (UniqueName: \"kubernetes.io/projected/ece299e0-879f-455d-af4f-ffa0f11df858-kube-api-access-9t59v\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093455    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smm95\" (UniqueName: \"kubernetes.io/projected/52a41276-3e4b-4199-ab64-387a2bc6a61d-kube-api-access-smm95\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.093628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.093463    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ece299e0-879f-455d-af4f-ffa0f11df858-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:30.145214 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145142    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:30.145527 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145514    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece299e0-879f-455d-af4f-ffa0f11df858" containerName="main"
Jul 30 17:38:30.145572 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145528    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece299e0-879f-455d-af4f-ffa0f11df858" containerName="main"
Jul 30 17:38:30.145572 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145544    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" containerName="main"
Jul 30 17:38:30.145572 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145549    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" containerName="main"
Jul 30 17:38:30.145572 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145563    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a41276-3e4b-4199-ab64-387a2bc6a61d" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:30.145572 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145569    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a41276-3e4b-4199-ab64-387a2bc6a61d" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:30.145738 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145645    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a41276-3e4b-4199-ab64-387a2bc6a61d" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:30.145738 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145654    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" containerName="main"
Jul 30 17:38:30.145738 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.145660    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece299e0-879f-455d-af4f-ffa0f11df858" containerName="main"
Jul 30 17:38:30.148695 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.148681    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.154319 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.154294    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:30.194694 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.194662    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwvh2\" (UniqueName: \"kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2\") pod \"test-curl-7785-140008135606320\" (UID: \"aed999af-8e97-47ab-adda-251bed07d86e\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.296010 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.295972    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwvh2\" (UniqueName: \"kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2\") pod \"test-curl-7785-140008135606320\" (UID: \"aed999af-8e97-47ab-adda-251bed07d86e\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.305196 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.305173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwvh2\" (UniqueName: \"kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2\") pod \"test-curl-7785-140008135606320\" (UID: \"aed999af-8e97-47ab-adda-251bed07d86e\") " pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.460852 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.460763    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.595732 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.595705    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:30.597469 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:30.597444    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaed999af_8e97_47ab_adda_251bed07d86e.slice/crio-8f2ea2788be952f91ed0483823db4baa3ce66a8811dff2c7be583d97af4d444c WatchSource:0}: Error finding container 8f2ea2788be952f91ed0483823db4baa3ce66a8811dff2c7be583d97af4d444c: Status 404 returned error can't find the container with id 8f2ea2788be952f91ed0483823db4baa3ce66a8811dff2c7be583d97af4d444c
Jul 30 17:38:30.843187 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.843145    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46" event={"ID":"ece299e0-879f-455d-af4f-ffa0f11df858","Type":"ContainerDied","Data":"f931dfedc7d559d44e600af353fece83fa8a32089035a174338797dc71067d60"}
Jul 30 17:38:30.843361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.843203    2561 scope.go:117] "RemoveContainer" containerID="66210f56652416d675b410d50514968245095f0cef3d261854de373a41fd7154"
Jul 30 17:38:30.843361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.843165    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"
Jul 30 17:38:30.844455 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.844427    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135606320" event={"ID":"aed999af-8e97-47ab-adda-251bed07d86e","Type":"ContainerStarted","Data":"8f2ea2788be952f91ed0483823db4baa3ce66a8811dff2c7be583d97af4d444c"}
Jul 30 17:38:30.845875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.845859    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:30.845995 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.845884    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e36daf9eb3a50a989a1214b14fbaea818573eef821b71c88a7b94e7caec29e35"
Jul 30 17:38:30.869780 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.869753    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:38:30.873544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.873520    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-27deb9/rate-test-model-27deb9-kserve-59c75c7d7-t9f46"]
Jul 30 17:38:30.874807 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:30.874778    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7785-140008135606320" oldPodUID="52a41276-3e4b-4199-ab64-387a2bc6a61d" podUID="aed999af-8e97-47ab-adda-251bed07d86e"
Jul 30 17:38:31.340473 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:31.340443    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a41276-3e4b-4199-ab64-387a2bc6a61d" path="/var/lib/kubelet/pods/52a41276-3e4b-4199-ab64-387a2bc6a61d/volumes"
Jul 30 17:38:31.340853 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:31.340831    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83bc52a3-d19a-4c58-9f7e-55d8fe312ba7" path="/var/lib/kubelet/pods/83bc52a3-d19a-4c58-9f7e-55d8fe312ba7/volumes"
Jul 30 17:38:31.341167 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:31.341154    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece299e0-879f-455d-af4f-ffa0f11df858" path="/var/lib/kubelet/pods/ece299e0-879f-455d-af4f-ffa0f11df858/volumes"
Jul 30 17:38:31.853285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:31.853252    2561 generic.go:356] "Generic (PLEG): container finished" podID="aed999af-8e97-47ab-adda-251bed07d86e" containerID="44978ba7fb5c42c5a2f53a47e3690aa9a1625f245bc4d8240297b8dea6c5ee50" exitCode=0
Jul 30 17:38:31.853436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:31.853297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135606320" event={"ID":"aed999af-8e97-47ab-adda-251bed07d86e","Type":"ContainerDied","Data":"44978ba7fb5c42c5a2f53a47e3690aa9a1625f245bc4d8240297b8dea6c5ee50"}
Jul 30 17:38:32.999269 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:32.999242    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:33.020136 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.020116    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135606320_aed999af-8e97-47ab-adda-251bed07d86e/test-curl-7785-140008135606320/0.log"
Jul 30 17:38:33.054080 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.054049    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:33.058133 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.058107    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135606320"]
Jul 30 17:38:33.119433 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.119354    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwvh2\" (UniqueName: \"kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2\") pod \"aed999af-8e97-47ab-adda-251bed07d86e\" (UID: \"aed999af-8e97-47ab-adda-251bed07d86e\") "
Jul 30 17:38:33.121406 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.121381    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2" (OuterVolumeSpecName: "kube-api-access-mwvh2") pod "aed999af-8e97-47ab-adda-251bed07d86e" (UID: "aed999af-8e97-47ab-adda-251bed07d86e"). InnerVolumeSpecName "kube-api-access-mwvh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:33.220438 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.220404    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwvh2\" (UniqueName: \"kubernetes.io/projected/aed999af-8e97-47ab-adda-251bed07d86e-kube-api-access-mwvh2\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:38:33.341058 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.341029    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed999af-8e97-47ab-adda-251bed07d86e" path="/var/lib/kubelet/pods/aed999af-8e97-47ab-adda-251bed07d86e/volumes"
Jul 30 17:38:33.868511 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.864340    2561 scope.go:117] "RemoveContainer" containerID="44978ba7fb5c42c5a2f53a47e3690aa9a1625f245bc4d8240297b8dea6c5ee50"
Jul 30 17:38:33.868511 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:33.864464    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135606320"
Jul 30 17:38:39.114967 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.114933    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:38:39.115350 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.115282    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed999af-8e97-47ab-adda-251bed07d86e" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:39.115350 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.115294    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed999af-8e97-47ab-adda-251bed07d86e" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:39.115434 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.115373    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed999af-8e97-47ab-adda-251bed07d86e" containerName="test-curl-7785-140008135606320"
Jul 30 17:38:39.119734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.119715    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.121989 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.121962    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d280ec\""
Jul 30 17:38:39.131326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.131295    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:38:39.141584 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.141554    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:38:39.145438 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.145417    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.148002 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.147983    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d280ec-dockercfg-bxmqm\""
Jul 30 17:38:39.152850 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.152828    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:38:39.166463 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.166439    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.166592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.166500    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrzn\" (UniqueName: \"kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.267094 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.267060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.267259 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.267112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrzn\" (UniqueName: \"kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.267259 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.267147    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.267259 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.267173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rchc\" (UniqueName: \"kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.267805 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.267786    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.275498 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.275463    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrzn\" (UniqueName: \"kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn\") pod \"payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.367799 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.367727    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.367799 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.367768    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rchc\" (UniqueName: \"kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.368473 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.368454    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.375600 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.375576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rchc\" (UniqueName: \"kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc\") pod \"payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.430522 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.430479    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.455700 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.455674    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.783182 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.783160    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:38:39.785666 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:39.785634    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78b01f9_c29c_4280_bd04_e3b20bc12b7a.slice/crio-9cdf890f258c983dc25314dc7ceea744d55fd4f6ed37cfcbf932dfa4e7a7d3fe WatchSource:0}: Error finding container 9cdf890f258c983dc25314dc7ceea744d55fd4f6ed37cfcbf932dfa4e7a7d3fe: Status 404 returned error can't find the container with id 9cdf890f258c983dc25314dc7ceea744d55fd4f6ed37cfcbf932dfa4e7a7d3fe
Jul 30 17:38:39.814616 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.814594    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:38:39.824654 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:38:39.824629    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04dc455_764b_4609_80b7_75a336054b7e.slice/crio-445aee1c6ea9f420f67e5f73985afe6e0d48c74c8dff027243091e6ee3cf705c WatchSource:0}: Error finding container 445aee1c6ea9f420f67e5f73985afe6e0d48c74c8dff027243091e6ee3cf705c: Status 404 returned error can't find the container with id 445aee1c6ea9f420f67e5f73985afe6e0d48c74c8dff027243091e6ee3cf705c
Jul 30 17:38:39.898650 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.898612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" event={"ID":"b04dc455-764b-4609-80b7-75a336054b7e","Type":"ContainerStarted","Data":"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"}
Jul 30 17:38:39.898809 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.898687    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:39.898809 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.898716    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" event={"ID":"b04dc455-764b-4609-80b7-75a336054b7e","Type":"ContainerStarted","Data":"445aee1c6ea9f420f67e5f73985afe6e0d48c74c8dff027243091e6ee3cf705c"}
Jul 30 17:38:39.900223 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.900200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" event={"ID":"d78b01f9-c29c-4280-bd04-e3b20bc12b7a","Type":"ContainerStarted","Data":"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"}
Jul 30 17:38:39.900351 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.900240    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" event={"ID":"d78b01f9-c29c-4280-bd04-e3b20bc12b7a","Type":"ContainerStarted","Data":"9cdf890f258c983dc25314dc7ceea744d55fd4f6ed37cfcbf932dfa4e7a7d3fe"}
Jul 30 17:38:39.900351 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.900329    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:38:39.916981 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.916935    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" podStartSLOduration=0.916920236 podStartE2EDuration="916.920236ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:39.915402767 +0000 UTC m=+3749.172864207" watchObservedRunningTime="2026-07-30 17:38:39.916920236 +0000 UTC m=+3749.174381688"
Jul 30 17:38:39.933329 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:39.933276    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" podStartSLOduration=0.933261738 podStartE2EDuration="933.261738ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:39.930944772 +0000 UTC m=+3749.188406192" watchObservedRunningTime="2026-07-30 17:38:39.933261738 +0000 UTC m=+3749.190723158"
Jul 30 17:38:45.423832 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:45.423794    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:38:50.907968 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:50.907938    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:38:50.908481 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:38:50.908175    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:39:08.268752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.268708    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:39:08.272553 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.272533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.275014 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.274990    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6e5025\""
Jul 30 17:39:08.275338 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.275313    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6e5025-dockercfg-rdpcd\""
Jul 30 17:39:08.287982 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.287952    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:39:08.425246 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.425212    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.425430 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.425289    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljvf8\" (UniqueName: \"kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.525953 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.525882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljvf8\" (UniqueName: \"kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.525953 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.525946    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.526600 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.526583    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.534498 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.534456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljvf8\" (UniqueName: \"kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8\") pod \"payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.583451 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.583420    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:08.717583 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:08.717558    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:39:08.719595 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:39:08.719554    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ce3eb3_fcc1_4302_b118_5efe97ed0959.slice/crio-d84280bb0b18199ce6b44b8881bd89fa7b22cf7d08a9a5a4258f4a996927c51f WatchSource:0}: Error finding container d84280bb0b18199ce6b44b8881bd89fa7b22cf7d08a9a5a4258f4a996927c51f: Status 404 returned error can't find the container with id d84280bb0b18199ce6b44b8881bd89fa7b22cf7d08a9a5a4258f4a996927c51f
Jul 30 17:39:09.051966 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:09.051876    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" event={"ID":"f8ce3eb3-fcc1-4302-b118-5efe97ed0959","Type":"ContainerStarted","Data":"5d3c2c6231f34b60d860e89bbfbf8db2d74a659e2c49d1f63238699d48d23760"}
Jul 30 17:39:09.051966 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:09.051930    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" event={"ID":"f8ce3eb3-fcc1-4302-b118-5efe97ed0959","Type":"ContainerStarted","Data":"d84280bb0b18199ce6b44b8881bd89fa7b22cf7d08a9a5a4258f4a996927c51f"}
Jul 30 17:39:09.052159 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:09.051969    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:09.074134 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:09.074078    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" podStartSLOduration=1.074057998 podStartE2EDuration="1.074057998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:09.073840597 +0000 UTC m=+3778.331302016" watchObservedRunningTime="2026-07-30 17:39:09.074057998 +0000 UTC m=+3778.331519424"
Jul 30 17:39:16.220390 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:16.220351    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:39:20.059637 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:20.059601    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:39:38.465238 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:38.465209    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw_f8ce3eb3-fcc1-4302-b118-5efe97ed0959/payload-processing/0.log"
Jul 30 17:39:45.838335 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.838302    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:39:45.842438 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.842417    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.847633 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.847610    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d280ec\"/\"kube-root-ca.crt\""
Jul 30 17:39:45.848559 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.848537    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d280ec\"/\"default-dockercfg-s5tw7\""
Jul 30 17:39:45.848559 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.848552    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d280ec\"/\"openshift-service-ca.crt\""
Jul 30 17:39:45.848704 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.848640    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d280ec\"/\"ipp-route-d280ec-kserve-self-signed-certs\""
Jul 30 17:39:45.854826 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.854802    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:39:45.933227 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933200    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.933378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933233    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.933378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.933378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933338    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.933378 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933368    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2szn4\" (UniqueName: \"kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:45.933565 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:45.933389    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.034705 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.034705 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2szn4\" (UniqueName: \"kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035030 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034733    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035030 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035030 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035030 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.034881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035272 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.035131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035349 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.035265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.035404 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.035374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.037270 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.037243    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.037374 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.037360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.043013 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.042993    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2szn4\" (UniqueName: \"kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4\") pod \"ipp-route-d280ec-kserve-6cf9b7868c-dklb7\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") " pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.151983 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.151884    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:46.289057 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:46.289024    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:39:46.292191 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:39:46.292164    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod406eec48_d663_4982_b32d_3b462c7a9e4e.slice/crio-fead497c501f3c93c3998c58e1ac5ee449285a90960cec19884605784637b94e WatchSource:0}: Error finding container fead497c501f3c93c3998c58e1ac5ee449285a90960cec19884605784637b94e: Status 404 returned error can't find the container with id fead497c501f3c93c3998c58e1ac5ee449285a90960cec19884605784637b94e
Jul 30 17:39:47.245326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:47.245290    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" event={"ID":"406eec48-d663-4982-b32d-3b462c7a9e4e","Type":"ContainerStarted","Data":"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"}
Jul 30 17:39:47.245326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:47.245329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" event={"ID":"406eec48-d663-4982-b32d-3b462c7a9e4e","Type":"ContainerStarted","Data":"fead497c501f3c93c3998c58e1ac5ee449285a90960cec19884605784637b94e"}
Jul 30 17:39:47.269343 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:47.269280    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" podStartSLOduration=2.269262355 podStartE2EDuration="2.269262355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:47.268766691 +0000 UTC m=+3816.526228112" watchObservedRunningTime="2026-07-30 17:39:47.269262355 +0000 UTC m=+3816.526723776"
Jul 30 17:39:50.123956 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:50.123924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:39:56.152743 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:56.152711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:56.153226 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:56.153061    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:56.167288 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:56.167262    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:39:56.301582 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:39:56.301559    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:40:18.321720 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:18.321687    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:40:28.345026 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:28.344980    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq_b04dc455-764b-4609-80b7-75a336054b7e/payload-processing/0.log"
Jul 30 17:40:28.595111 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:28.595036    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw_f8ce3eb3-fcc1-4302-b118-5efe97ed0959/payload-processing/0.log"
Jul 30 17:40:33.815604 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.815525    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:40:33.819274 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.819253    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.822209 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.822184    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ea2bbf\""
Jul 30 17:40:33.839209 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.839180    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nxwb\" (UniqueName: \"kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.839316 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.839299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.853976 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.853954    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:40:33.864547 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.864523    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:40:33.868050 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.868036    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:33.870782 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.870759    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ea2bbf-dockercfg-68jgz\""
Jul 30 17:40:33.894838 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.894811    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:40:33.939807 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.939773    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nxwb\" (UniqueName: \"kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.939977 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.939914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbn9f\" (UniqueName: \"kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:33.939977 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.939965    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:33.940087 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.940005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.940933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.940906    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:33.952592 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:33.952563    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nxwb\" (UniqueName: \"kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:34.040686 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.040653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbn9f\" (UniqueName: \"kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:34.040858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.040697    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:34.041384 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.041364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:34.053465 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.053443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbn9f\" (UniqueName: \"kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f\") pod \"payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:34.129766 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.129729    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:34.178436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.178383    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:34.278064 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.277648    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:40:34.280510 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:40:34.280050    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf751c35f_714e_4428_8e5f_764f45e089cf.slice/crio-a99d355eac18189d9c746918ad2fda9a0c3adf74fc0f2400580bc32b83da1caa WatchSource:0}: Error finding container a99d355eac18189d9c746918ad2fda9a0c3adf74fc0f2400580bc32b83da1caa: Status 404 returned error can't find the container with id a99d355eac18189d9c746918ad2fda9a0c3adf74fc0f2400580bc32b83da1caa
Jul 30 17:40:34.478320 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.478227    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" event={"ID":"f751c35f-714e-4428-8e5f-764f45e089cf","Type":"ContainerStarted","Data":"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"}
Jul 30 17:40:34.478320 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.478279    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" event={"ID":"f751c35f-714e-4428-8e5f-764f45e089cf","Type":"ContainerStarted","Data":"a99d355eac18189d9c746918ad2fda9a0c3adf74fc0f2400580bc32b83da1caa"}
Jul 30 17:40:34.478320 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.478308    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:34.495427 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.495374    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" podStartSLOduration=1.495352226 podStartE2EDuration="1.495352226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:34.494292105 +0000 UTC m=+3863.751753552" watchObservedRunningTime="2026-07-30 17:40:34.495352226 +0000 UTC m=+3863.752813647"
Jul 30 17:40:34.537916 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:34.537826    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:40:34.540142 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:40:34.540115    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5213d4f9_97b4_47d8_bb4d_b8c05fd82c2b.slice/crio-c56b1b1e236f09013db9ea401727234cd4e0bdeb873b0c50ead90cb6c433aedf WatchSource:0}: Error finding container c56b1b1e236f09013db9ea401727234cd4e0bdeb873b0c50ead90cb6c433aedf: Status 404 returned error can't find the container with id c56b1b1e236f09013db9ea401727234cd4e0bdeb873b0c50ead90cb6c433aedf
Jul 30 17:40:35.485339 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:35.485303    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" event={"ID":"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b","Type":"ContainerStarted","Data":"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"}
Jul 30 17:40:35.485784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:35.485366    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" event={"ID":"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b","Type":"ContainerStarted","Data":"c56b1b1e236f09013db9ea401727234cd4e0bdeb873b0c50ead90cb6c433aedf"}
Jul 30 17:40:35.485784 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:35.485515    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:40:35.502935 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:35.502883    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" podStartSLOduration=2.502869984 podStartE2EDuration="2.502869984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:35.50073673 +0000 UTC m=+3864.758198174" watchObservedRunningTime="2026-07-30 17:40:35.502869984 +0000 UTC m=+3864.760331469"
Jul 30 17:40:38.218292 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:38.218258    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:40:45.487646 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:45.487616    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:40:46.492522 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:40:46.492453    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:41:03.637649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.637614    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:41:03.638139 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.637871    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" podUID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" containerName="payload-processing" containerID="cri-o://40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e" gracePeriod=30
Jul 30 17:41:03.647009 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.646980    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:41:03.647718 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.647546    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" podUID="f751c35f-714e-4428-8e5f-764f45e089cf" containerName="payload-pre-processing" containerID="cri-o://3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501" gracePeriod=30
Jul 30 17:41:03.923513 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.923477    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:41:03.926541 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:03.926523    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:41:04.006414 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006376    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nxwb\" (UniqueName: \"kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb\") pod \"f751c35f-714e-4428-8e5f-764f45e089cf\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") "
Jul 30 17:41:04.006603 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006421    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume\") pod \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") "
Jul 30 17:41:04.006603 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006472    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume\") pod \"f751c35f-714e-4428-8e5f-764f45e089cf\" (UID: \"f751c35f-714e-4428-8e5f-764f45e089cf\") "
Jul 30 17:41:04.006603 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006585    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbn9f\" (UniqueName: \"kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f\") pod \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\" (UID: \"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b\") "
Jul 30 17:41:04.006872 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006843    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" (UID: "5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:04.006983 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006911    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.006983 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.006957    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f751c35f-714e-4428-8e5f-764f45e089cf" (UID: "f751c35f-714e-4428-8e5f-764f45e089cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:04.008772 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.008751    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f" (OuterVolumeSpecName: "kube-api-access-jbn9f") pod "5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" (UID: "5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b"). InnerVolumeSpecName "kube-api-access-jbn9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:04.008861 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.008798    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb" (OuterVolumeSpecName: "kube-api-access-2nxwb") pod "f751c35f-714e-4428-8e5f-764f45e089cf" (UID: "f751c35f-714e-4428-8e5f-764f45e089cf"). InnerVolumeSpecName "kube-api-access-2nxwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:04.107692 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.107661    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbn9f\" (UniqueName: \"kubernetes.io/projected/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b-kube-api-access-jbn9f\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.107692 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.107689    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nxwb\" (UniqueName: \"kubernetes.io/projected/f751c35f-714e-4428-8e5f-764f45e089cf-kube-api-access-2nxwb\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.107692 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.107699    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f751c35f-714e-4428-8e5f-764f45e089cf-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.635885 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.635849    2561 generic.go:356] "Generic (PLEG): container finished" podID="f751c35f-714e-4428-8e5f-764f45e089cf" containerID="3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501" exitCode=0
Jul 30 17:41:04.636077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.635892    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"
Jul 30 17:41:04.636077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.635928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" event={"ID":"f751c35f-714e-4428-8e5f-764f45e089cf","Type":"ContainerDied","Data":"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"}
Jul 30 17:41:04.636077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.635981    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw" event={"ID":"f751c35f-714e-4428-8e5f-764f45e089cf","Type":"ContainerDied","Data":"a99d355eac18189d9c746918ad2fda9a0c3adf74fc0f2400580bc32b83da1caa"}
Jul 30 17:41:04.636077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.636005    2561 scope.go:117] "RemoveContainer" containerID="3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"
Jul 30 17:41:04.637208 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.637188    2561 generic.go:356] "Generic (PLEG): container finished" podID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" containerID="40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e" exitCode=0
Jul 30 17:41:04.637308 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.637235    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"
Jul 30 17:41:04.637308 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.637268    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" event={"ID":"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b","Type":"ContainerDied","Data":"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"}
Jul 30 17:41:04.637418 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.637308    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg" event={"ID":"5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b","Type":"ContainerDied","Data":"c56b1b1e236f09013db9ea401727234cd4e0bdeb873b0c50ead90cb6c433aedf"}
Jul 30 17:41:04.647048 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.646859    2561 scope.go:117] "RemoveContainer" containerID="3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"
Jul 30 17:41:04.652562 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:41:04.647617    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501\": container with ID starting with 3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501 not found: ID does not exist" containerID="3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"
Jul 30 17:41:04.652562 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.647705    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501"} err="failed to get container status \"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501\": rpc error: code = NotFound desc = could not find container \"3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501\": container with ID starting with 3e253e7e145c5c0d8dcc53b647acc8e1677136552bd0fc161db08e9f90737501 not found: ID does not exist"
Jul 30 17:41:04.652562 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.647753    2561 scope.go:117] "RemoveContainer" containerID="40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"
Jul 30 17:41:04.658972 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.658947    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:41:04.664434 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.664409    2561 scope.go:117] "RemoveContainer" containerID="40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"
Jul 30 17:41:04.664751 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.664732    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea2bbf-7ff8bffb44-z7sxw"]
Jul 30 17:41:04.664751 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:41:04.664738    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e\": container with ID starting with 40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e not found: ID does not exist" containerID="40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"
Jul 30 17:41:04.664875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.664766    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e"} err="failed to get container status \"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e\": rpc error: code = NotFound desc = could not find container \"40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e\": container with ID starting with 40433d3bd533274b833a8a95e3606c5e0b2f917a1653a3820fec37ca009f265e not found: ID does not exist"
Jul 30 17:41:04.678833 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.678814    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:41:04.683917 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:04.683897    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea2bbf-7549b888-sqqpg"]
Jul 30 17:41:05.342308 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:05.342274    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" path="/var/lib/kubelet/pods/5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b/volumes"
Jul 30 17:41:05.342649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:05.342634    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f751c35f-714e-4428-8e5f-764f45e089cf" path="/var/lib/kubelet/pods/f751c35f-714e-4428-8e5f-764f45e089cf/volumes"
Jul 30 17:41:08.121362 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:08.121323    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:41:11.488424 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:11.488399    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:41:11.494536 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:11.494513    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:41:11.494969 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:11.494940    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:41:11.500650 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:11.500635    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrnv9_de4fa743-0ef2-470f-b389-8eb9516cca1d/ovn-acl-logging/0.log"
Jul 30 17:41:28.523606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:28.523574    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:41:29.703807 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.703760    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:41:29.704233 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.704031    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" podUID="b04dc455-764b-4609-80b7-75a336054b7e" containerName="payload-processing" containerID="cri-o://7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc" gracePeriod=30
Jul 30 17:41:29.710700 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.710669    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:41:29.710979 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.710952    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" podUID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" containerName="payload-pre-processing" containerID="cri-o://4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31" gracePeriod=30
Jul 30 17:41:29.991198 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.991174    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:41:29.993895 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:29.993878    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:41:30.016074 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016052    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume\") pod \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") "
Jul 30 17:41:30.016197 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016144    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume\") pod \"b04dc455-764b-4609-80b7-75a336054b7e\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") "
Jul 30 17:41:30.016197 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016164    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rchc\" (UniqueName: \"kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc\") pod \"b04dc455-764b-4609-80b7-75a336054b7e\" (UID: \"b04dc455-764b-4609-80b7-75a336054b7e\") "
Jul 30 17:41:30.016197 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016184    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vrzn\" (UniqueName: \"kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn\") pod \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\" (UID: \"d78b01f9-c29c-4280-bd04-e3b20bc12b7a\") "
Jul 30 17:41:30.016454 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016433    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d78b01f9-c29c-4280-bd04-e3b20bc12b7a" (UID: "d78b01f9-c29c-4280-bd04-e3b20bc12b7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:30.016620 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.016600    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b04dc455-764b-4609-80b7-75a336054b7e" (UID: "b04dc455-764b-4609-80b7-75a336054b7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:30.018292 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.018259    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc" (OuterVolumeSpecName: "kube-api-access-5rchc") pod "b04dc455-764b-4609-80b7-75a336054b7e" (UID: "b04dc455-764b-4609-80b7-75a336054b7e"). InnerVolumeSpecName "kube-api-access-5rchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:30.018402 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.018362    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn" (OuterVolumeSpecName: "kube-api-access-6vrzn") pod "d78b01f9-c29c-4280-bd04-e3b20bc12b7a" (UID: "d78b01f9-c29c-4280-bd04-e3b20bc12b7a"). InnerVolumeSpecName "kube-api-access-6vrzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:30.116858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.116824    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b04dc455-764b-4609-80b7-75a336054b7e-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.116858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.116854    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rchc\" (UniqueName: \"kubernetes.io/projected/b04dc455-764b-4609-80b7-75a336054b7e-kube-api-access-5rchc\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.116858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.116864    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vrzn\" (UniqueName: \"kubernetes.io/projected/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-kube-api-access-6vrzn\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.116858 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.116873    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b01f9-c29c-4280-bd04-e3b20bc12b7a-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.783837 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.783800    2561 generic.go:356] "Generic (PLEG): container finished" podID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" containerID="4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31" exitCode=0
Jul 30 17:41:30.784283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.783843    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"
Jul 30 17:41:30.784283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.783882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" event={"ID":"d78b01f9-c29c-4280-bd04-e3b20bc12b7a","Type":"ContainerDied","Data":"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"}
Jul 30 17:41:30.784283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.783936    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt" event={"ID":"d78b01f9-c29c-4280-bd04-e3b20bc12b7a","Type":"ContainerDied","Data":"9cdf890f258c983dc25314dc7ceea744d55fd4f6ed37cfcbf932dfa4e7a7d3fe"}
Jul 30 17:41:30.784283 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.783960    2561 scope.go:117] "RemoveContainer" containerID="4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"
Jul 30 17:41:30.785478 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.785455    2561 generic.go:356] "Generic (PLEG): container finished" podID="b04dc455-764b-4609-80b7-75a336054b7e" containerID="7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc" exitCode=0
Jul 30 17:41:30.785606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.785518    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"
Jul 30 17:41:30.785606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.785533    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" event={"ID":"b04dc455-764b-4609-80b7-75a336054b7e","Type":"ContainerDied","Data":"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"}
Jul 30 17:41:30.785606 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.785575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq" event={"ID":"b04dc455-764b-4609-80b7-75a336054b7e","Type":"ContainerDied","Data":"445aee1c6ea9f420f67e5f73985afe6e0d48c74c8dff027243091e6ee3cf705c"}
Jul 30 17:41:30.794536 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.794411    2561 scope.go:117] "RemoveContainer" containerID="4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"
Jul 30 17:41:30.794777 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:41:30.794749    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31\": container with ID starting with 4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31 not found: ID does not exist" containerID="4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"
Jul 30 17:41:30.794854 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.794784    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31"} err="failed to get container status \"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31\": rpc error: code = NotFound desc = could not find container \"4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31\": container with ID starting with 4cda50a8a837a3db8227356f67d5fd6a6a522b75af38fb41e986075b8aeb9c31 not found: ID does not exist"
Jul 30 17:41:30.794854 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.794802    2561 scope.go:117] "RemoveContainer" containerID="7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"
Jul 30 17:41:30.803173 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.803155    2561 scope.go:117] "RemoveContainer" containerID="7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"
Jul 30 17:41:30.803410 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:41:30.803392    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc\": container with ID starting with 7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc not found: ID does not exist" containerID="7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"
Jul 30 17:41:30.803477 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.803419    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc"} err="failed to get container status \"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc\": rpc error: code = NotFound desc = could not find container \"7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc\": container with ID starting with 7da59e15de3c4f66435bcdcf0c1e0783e35a3bb24e29bddd153598f5a492c8fc not found: ID does not exist"
Jul 30 17:41:30.810225 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.810200    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:41:30.814363 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.814342    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d280ec-9c7b9fc7d-bp9zq"]
Jul 30 17:41:30.828330 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.828307    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:41:30.834463 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:30.834440    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d280ec-68b6ccdc6d-8t8rt"]
Jul 30 17:41:31.341384 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:31.341352    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04dc455-764b-4609-80b7-75a336054b7e" path="/var/lib/kubelet/pods/b04dc455-764b-4609-80b7-75a336054b7e/volumes"
Jul 30 17:41:31.341728 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:31.341715    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" path="/var/lib/kubelet/pods/d78b01f9-c29c-4280-bd04-e3b20bc12b7a/volumes"
Jul 30 17:41:35.199290 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:35.199249    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:41:41.417907 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:41.417867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:41:50.218649 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.218603    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" podUID="406eec48-d663-4982-b32d-3b462c7a9e4e" containerName="main" containerID="cri-o://3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae" gracePeriod=45
Jul 30 17:41:50.470207 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.470136    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:41:50.581371 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581341    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581371 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581406    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581433    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581471    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2szn4\" (UniqueName: \"kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581635 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs\") pod \"406eec48-d663-4982-b32d-3b462c7a9e4e\" (UID: \"406eec48-d663-4982-b32d-3b462c7a9e4e\") "
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581676    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home" (OuterVolumeSpecName: "home") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581691    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581683    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581780    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-home\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581801    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-tmp-dir\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.581856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.581815    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-model-cache\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.583681 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.583647    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm" (OuterVolumeSpecName: "dshm") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:41:50.583791 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.583730    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4" (OuterVolumeSpecName: "kube-api-access-2szn4") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "kube-api-access-2szn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:50.583791 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.583741    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "406eec48-d663-4982-b32d-3b462c7a9e4e" (UID: "406eec48-d663-4982-b32d-3b462c7a9e4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:41:50.682890 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.682864    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406eec48-d663-4982-b32d-3b462c7a9e4e-dshm\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.682890 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.682887    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2szn4\" (UniqueName: \"kubernetes.io/projected/406eec48-d663-4982-b32d-3b462c7a9e4e-kube-api-access-2szn4\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.682890 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.682897    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406eec48-d663-4982-b32d-3b462c7a9e4e-tls-certs\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:41:50.893399 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.893365    2561 generic.go:356] "Generic (PLEG): container finished" podID="406eec48-d663-4982-b32d-3b462c7a9e4e" containerID="3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae" exitCode=0
Jul 30 17:41:50.893613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.893411    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"
Jul 30 17:41:50.893613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.893449    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" event={"ID":"406eec48-d663-4982-b32d-3b462c7a9e4e","Type":"ContainerDied","Data":"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"}
Jul 30 17:41:50.893613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.893502    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7" event={"ID":"406eec48-d663-4982-b32d-3b462c7a9e4e","Type":"ContainerDied","Data":"fead497c501f3c93c3998c58e1ac5ee449285a90960cec19884605784637b94e"}
Jul 30 17:41:50.893613 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.893519    2561 scope.go:117] "RemoveContainer" containerID="3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"
Jul 30 17:41:50.902667 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.902651    2561 scope.go:117] "RemoveContainer" containerID="3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"
Jul 30 17:41:50.902899 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:41:50.902882    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae\": container with ID starting with 3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae not found: ID does not exist" containerID="3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"
Jul 30 17:41:50.902969 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.902906    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae"} err="failed to get container status \"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae\": rpc error: code = NotFound desc = could not find container \"3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae\": container with ID starting with 3855d0672c4e45a4c456d664d4047ef527958e695b00de345d1d4cdeeeaaddae not found: ID does not exist"
Jul 30 17:41:50.914988 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.914960    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:41:50.918249 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:50.918230    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-d280ec/ipp-route-d280ec-kserve-6cf9b7868c-dklb7"]
Jul 30 17:41:51.341210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:41:51.341182    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406eec48-d663-4982-b32d-3b462c7a9e4e" path="/var/lib/kubelet/pods/406eec48-d663-4982-b32d-3b462c7a9e4e/volumes"
Jul 30 17:42:33.364595 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364561    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"]
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364926    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04dc455-764b-4609-80b7-75a336054b7e" containerName="payload-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364937    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04dc455-764b-4609-80b7-75a336054b7e" containerName="payload-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364952    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364958    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364965    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" containerName="payload-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364972    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" containerName="payload-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364983    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f751c35f-714e-4428-8e5f-764f45e089cf" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364989    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f751c35f-714e-4428-8e5f-764f45e089cf" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.364998    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406eec48-d663-4982-b32d-3b462c7a9e4e" containerName="main"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365003    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="406eec48-d663-4982-b32d-3b462c7a9e4e" containerName="main"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365060    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="406eec48-d663-4982-b32d-3b462c7a9e4e" containerName="main"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365068    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78b01f9-c29c-4280-bd04-e3b20bc12b7a" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365075    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5213d4f9-97b4-47d8-bb4d-b8c05fd82c2b" containerName="payload-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365082    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f751c35f-714e-4428-8e5f-764f45e089cf" containerName="payload-pre-processing"
Jul 30 17:42:33.367029 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.365087    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04dc455-764b-4609-80b7-75a336054b7e" containerName="payload-processing"
Jul 30 17:42:33.367813 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.367797    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:33.371076 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.371054    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:42:33.371332 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.371317    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:42:33.377951 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.377929    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"]
Jul 30 17:42:33.437623 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.437594    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr\" (UID: \"af134f21-742e-4bb3-927c-f06ec0603672\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:33.538130 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.538098    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr\" (UID: \"af134f21-742e-4bb3-927c-f06ec0603672\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:33.538728 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.538711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr\" (UID: \"af134f21-742e-4bb3-927c-f06ec0603672\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:33.679358 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.679277    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:33.815329 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:33.815297    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"]
Jul 30 17:42:33.817676 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:42:33.817650    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf134f21_742e_4bb3_927c_f06ec0603672.slice/crio-7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5 WatchSource:0}: Error finding container 7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5: Status 404 returned error can't find the container with id 7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5
Jul 30 17:42:34.106545 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:34.106516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr" event={"ID":"af134f21-742e-4bb3-927c-f06ec0603672","Type":"ContainerStarted","Data":"62c911d7b7b29e0cd5206ed1b1b259e3f30228973b258efc0319463ff048e9b4"}
Jul 30 17:42:34.106680 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:34.106570    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr" event={"ID":"af134f21-742e-4bb3-927c-f06ec0603672","Type":"ContainerStarted","Data":"7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5"}
Jul 30 17:42:34.125554 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:34.125481    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr" podStartSLOduration=1.125465109 podStartE2EDuration="1.125465109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:42:34.124844579 +0000 UTC m=+3983.382306022" watchObservedRunningTime="2026-07-30 17:42:34.125465109 +0000 UTC m=+3983.382926529"
Jul 30 17:42:35.113176 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:35.113135    2561 generic.go:356] "Generic (PLEG): container finished" podID="af134f21-742e-4bb3-927c-f06ec0603672" containerID="62c911d7b7b29e0cd5206ed1b1b259e3f30228973b258efc0319463ff048e9b4" exitCode=0
Jul 30 17:42:35.113619 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:35.113211    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr" event={"ID":"af134f21-742e-4bb3-927c-f06ec0603672","Type":"ContainerDied","Data":"62c911d7b7b29e0cd5206ed1b1b259e3f30228973b258efc0319463ff048e9b4"}
Jul 30 17:42:36.250031 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:36.250000    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:36.262587 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:36.262565    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca\") pod \"af134f21-742e-4bb3-927c-f06ec0603672\" (UID: \"af134f21-742e-4bb3-927c-f06ec0603672\") "
Jul 30 17:42:36.262850 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:36.262830    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "af134f21-742e-4bb3-927c-f06ec0603672" (UID: "af134f21-742e-4bb3-927c-f06ec0603672"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:42:36.363367 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:36.363329    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af134f21-742e-4bb3-927c-f06ec0603672-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:42:37.123946 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:37.123915    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"
Jul 30 17:42:37.124122 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:37.123958    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr" event={"ID":"af134f21-742e-4bb3-927c-f06ec0603672","Type":"ContainerDied","Data":"7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5"}
Jul 30 17:42:37.124122 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:37.124006    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a7b6c82d0207511dd441159f0d24b2081612fbcd0c0a72099fe2b39a5256ef5"
Jul 30 17:42:43.955971 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:43.955931    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:42:43.956512 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:43.956185    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" podUID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" containerName="payload-processing" containerID="cri-o://5d3c2c6231f34b60d860e89bbfbf8db2d74a659e2c49d1f63238699d48d23760" gracePeriod=30
Jul 30 17:42:44.115607 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.115567    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"]
Jul 30 17:42:44.119341 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.119311    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6e5025-8mrxr"]
Jul 30 17:42:44.164164 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.164122    2561 generic.go:356] "Generic (PLEG): container finished" podID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" containerID="5d3c2c6231f34b60d860e89bbfbf8db2d74a659e2c49d1f63238699d48d23760" exitCode=0
Jul 30 17:42:44.164340 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.164189    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" event={"ID":"f8ce3eb3-fcc1-4302-b118-5efe97ed0959","Type":"ContainerDied","Data":"5d3c2c6231f34b60d860e89bbfbf8db2d74a659e2c49d1f63238699d48d23760"}
Jul 30 17:42:44.220155 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.220093    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:42:44.331856 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.331815    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljvf8\" (UniqueName: \"kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8\") pod \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") "
Jul 30 17:42:44.332045 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.331877    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume\") pod \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\" (UID: \"f8ce3eb3-fcc1-4302-b118-5efe97ed0959\") "
Jul 30 17:42:44.332231 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.332206    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8ce3eb3-fcc1-4302-b118-5efe97ed0959" (UID: "f8ce3eb3-fcc1-4302-b118-5efe97ed0959"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:42:44.333963 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.333934    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8" (OuterVolumeSpecName: "kube-api-access-ljvf8") pod "f8ce3eb3-fcc1-4302-b118-5efe97ed0959" (UID: "f8ce3eb3-fcc1-4302-b118-5efe97ed0959"). InnerVolumeSpecName "kube-api-access-ljvf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:42:44.432596 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.432562    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljvf8\" (UniqueName: \"kubernetes.io/projected/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-kube-api-access-ljvf8\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.432596 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:44.432592    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ce3eb3-fcc1-4302-b118-5efe97ed0959-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:42:45.170444 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.170418    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"
Jul 30 17:42:45.170911 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.170413    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw" event={"ID":"f8ce3eb3-fcc1-4302-b118-5efe97ed0959","Type":"ContainerDied","Data":"d84280bb0b18199ce6b44b8881bd89fa7b22cf7d08a9a5a4258f4a996927c51f"}
Jul 30 17:42:45.170911 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.170579    2561 scope.go:117] "RemoveContainer" containerID="5d3c2c6231f34b60d860e89bbfbf8db2d74a659e2c49d1f63238699d48d23760"
Jul 30 17:42:45.193366 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.193343    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:42:45.197372 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.197352    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6e5025-6b6df59d98-wl2dw"]
Jul 30 17:42:45.340835 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.340802    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af134f21-742e-4bb3-927c-f06ec0603672" path="/var/lib/kubelet/pods/af134f21-742e-4bb3-927c-f06ec0603672/volumes"
Jul 30 17:42:45.341138 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:45.341125    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" path="/var/lib/kubelet/pods/f8ce3eb3-fcc1-4302-b118-5efe97ed0959/volumes"
Jul 30 17:42:47.324304 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:47.324269    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:42:50.838798 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.838763    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"]
Jul 30 17:42:50.839178 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839137    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" containerName="payload-processing"
Jul 30 17:42:50.839178 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839148    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" containerName="payload-processing"
Jul 30 17:42:50.839178 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839172    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af134f21-742e-4bb3-927c-f06ec0603672" containerName="revoke-keys"
Jul 30 17:42:50.839178 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839178    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="af134f21-742e-4bb3-927c-f06ec0603672" containerName="revoke-keys"
Jul 30 17:42:50.839326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839231    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ce3eb3-fcc1-4302-b118-5efe97ed0959" containerName="payload-processing"
Jul 30 17:42:50.839326 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.839239    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="af134f21-742e-4bb3-927c-f06ec0603672" containerName="revoke-keys"
Jul 30 17:42:50.843575 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.843558    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:50.846304 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.846273    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:42:50.847092 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.847067    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:42:50.850205 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.849808    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"]
Jul 30 17:42:50.891643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.891612    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f45123-hxspv\" (UID: \"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:50.992960 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.992929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f45123-hxspv\" (UID: \"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:50.993646 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:50.993624    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f45123-hxspv\" (UID: \"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:51.155510 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:51.155383    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:51.497208 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:51.497131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"]
Jul 30 17:42:51.499477 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:42:51.499447    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2823cac3_dd6b_42c9_8cc1_1c2c876fd0c6.slice/crio-9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747 WatchSource:0}: Error finding container 9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747: Status 404 returned error can't find the container with id 9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747
Jul 30 17:42:52.209050 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:52.209022    2561 generic.go:356] "Generic (PLEG): container finished" podID="2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" containerID="6be8d38e3f23073fe80ae1b9325e60b56fe144b6244fdb80ebe28480abd4bc71" exitCode=0
Jul 30 17:42:52.209396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:52.209054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv" event={"ID":"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6","Type":"ContainerDied","Data":"6be8d38e3f23073fe80ae1b9325e60b56fe144b6244fdb80ebe28480abd4bc71"}
Jul 30 17:42:52.209396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:52.209113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv" event={"ID":"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6","Type":"ContainerStarted","Data":"9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747"}
Jul 30 17:42:53.355149 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:53.355121    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:53.412827 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:53.412793    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca\") pod \"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6\" (UID: \"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6\") "
Jul 30 17:42:53.413137 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:53.413116    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" (UID: "2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:42:53.513584 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:53.513470    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6-maas-api-service-ca\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:42:54.223309 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:54.223275    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"
Jul 30 17:42:54.223499 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:54.223314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv" event={"ID":"2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6","Type":"ContainerDied","Data":"9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747"}
Jul 30 17:42:54.223499 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:42:54.223355    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d893de4dc5dbfe717813a213c44c7a1eb3375ee54d7b4b7ab3d28f70a041747"
Jul 30 17:43:01.373734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.373701    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:43:01.374647 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.373966    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" podUID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" containerName="payload-pre-processing" containerID="cri-o://444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825" gracePeriod=30
Jul 30 17:43:01.530440 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.530410    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"]
Jul 30 17:43:01.538628 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.533980    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f45123-hxspv"]
Jul 30 17:43:01.631734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.631680    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:43:01.792437 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.792401    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume\") pod \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") "
Jul 30 17:43:01.792651 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.792452    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgx6d\" (UniqueName: \"kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d\") pod \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\" (UID: \"9dda4bbe-fb6c-4833-b759-efe57cfdbc46\") "
Jul 30 17:43:01.792886 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.792860    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9dda4bbe-fb6c-4833-b759-efe57cfdbc46" (UID: "9dda4bbe-fb6c-4833-b759-efe57cfdbc46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:01.794667 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.794638    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d" (OuterVolumeSpecName: "kube-api-access-wgx6d") pod "9dda4bbe-fb6c-4833-b759-efe57cfdbc46" (UID: "9dda4bbe-fb6c-4833-b759-efe57cfdbc46"). InnerVolumeSpecName "kube-api-access-wgx6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:01.893246 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.893165    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.893246 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:01.893193    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgx6d\" (UniqueName: \"kubernetes.io/projected/9dda4bbe-fb6c-4833-b759-efe57cfdbc46-kube-api-access-wgx6d\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:02.265368 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.265284    2561 generic.go:356] "Generic (PLEG): container finished" podID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" containerID="444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825" exitCode=0
Jul 30 17:43:02.265368 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.265328    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"
Jul 30 17:43:02.265571 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.265376    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" event={"ID":"9dda4bbe-fb6c-4833-b759-efe57cfdbc46","Type":"ContainerDied","Data":"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"}
Jul 30 17:43:02.265571 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.265470    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4" event={"ID":"9dda4bbe-fb6c-4833-b759-efe57cfdbc46","Type":"ContainerDied","Data":"ddf090d2416603bfd715252b1dcd19580c1e0906753061fc51b8dc24e6f3c15a"}
Jul 30 17:43:02.265571 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.265516    2561 scope.go:117] "RemoveContainer" containerID="444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"
Jul 30 17:43:02.275199 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.275175    2561 scope.go:117] "RemoveContainer" containerID="444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"
Jul 30 17:43:02.275455 ip-10-0-132-133 kubenswrapper[2561]: E0730 17:43:02.275435    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825\": container with ID starting with 444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825 not found: ID does not exist" containerID="444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"
Jul 30 17:43:02.275568 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.275466    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825"} err="failed to get container status \"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825\": rpc error: code = NotFound desc = could not find container \"444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825\": container with ID starting with 444449b32920ca2fb326b93c6ad603283b5f971d43a4c2ad04d718b867e08825 not found: ID does not exist"
Jul 30 17:43:02.287404 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.287382    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:43:02.290580 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:02.290548    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f45123-7748cd9b5b-7j5t4"]
Jul 30 17:43:03.340840 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:03.340811    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" path="/var/lib/kubelet/pods/2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6/volumes"
Jul 30 17:43:03.341253 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:03.341138    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" path="/var/lib/kubelet/pods/9dda4bbe-fb6c-4833-b759-efe57cfdbc46/volumes"
Jul 30 17:43:04.523924 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:04.523889    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:43:19.147847 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.147729    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:43:19.148361 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.148006    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" podUID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" containerName="payload-processing" containerID="cri-o://cc6c2c07675185d7482e28dafef86a2787d8c8d33f31f07b55724d4d5541eed8" gracePeriod=30
Jul 30 17:43:19.154747 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.154720    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:43:19.155028 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.154978    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" podUID="93e8d615-a180-4053-8431-e12fa57229c6" containerName="payload-pre-processing" containerID="cri-o://14ae4217c588a6b2b8626102ab866cb1a0ef1fc84e8befc139a061fd243f1c2f" gracePeriod=30
Jul 30 17:43:19.366220 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.365988    2561 generic.go:356] "Generic (PLEG): container finished" podID="93e8d615-a180-4053-8431-e12fa57229c6" containerID="14ae4217c588a6b2b8626102ab866cb1a0ef1fc84e8befc139a061fd243f1c2f" exitCode=0
Jul 30 17:43:19.366220 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.366130    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" event={"ID":"93e8d615-a180-4053-8431-e12fa57229c6","Type":"ContainerDied","Data":"14ae4217c588a6b2b8626102ab866cb1a0ef1fc84e8befc139a061fd243f1c2f"}
Jul 30 17:43:19.368605 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.368575    2561 generic.go:356] "Generic (PLEG): container finished" podID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" containerID="cc6c2c07675185d7482e28dafef86a2787d8c8d33f31f07b55724d4d5541eed8" exitCode=0
Jul 30 17:43:19.368751 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.368618    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" event={"ID":"d9b0103e-aed1-4ce0-8c7c-234f7563100f","Type":"ContainerDied","Data":"cc6c2c07675185d7482e28dafef86a2787d8c8d33f31f07b55724d4d5541eed8"}
Jul 30 17:43:19.441096 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.441067    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:43:19.449245 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.449219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume\") pod \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") "
Jul 30 17:43:19.449358 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.449289    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9lmf\" (UniqueName: \"kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf\") pod \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\" (UID: \"d9b0103e-aed1-4ce0-8c7c-234f7563100f\") "
Jul 30 17:43:19.449638 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.449617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9b0103e-aed1-4ce0-8c7c-234f7563100f" (UID: "d9b0103e-aed1-4ce0-8c7c-234f7563100f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:19.451434 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.451413    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf" (OuterVolumeSpecName: "kube-api-access-g9lmf") pod "d9b0103e-aed1-4ce0-8c7c-234f7563100f" (UID: "d9b0103e-aed1-4ce0-8c7c-234f7563100f"). InnerVolumeSpecName "kube-api-access-g9lmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:19.454761 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.454742    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:43:19.550436 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.550407    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume\") pod \"93e8d615-a180-4053-8431-e12fa57229c6\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") "
Jul 30 17:43:19.550618 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.550601    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b0103e-aed1-4ce0-8c7c-234f7563100f-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.550618 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.550613    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9lmf\" (UniqueName: \"kubernetes.io/projected/d9b0103e-aed1-4ce0-8c7c-234f7563100f-kube-api-access-g9lmf\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.550735 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.550671    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93e8d615-a180-4053-8431-e12fa57229c6" (UID: "93e8d615-a180-4053-8431-e12fa57229c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:19.650905 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.650869    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9t4f\" (UniqueName: \"kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f\") pod \"93e8d615-a180-4053-8431-e12fa57229c6\" (UID: \"93e8d615-a180-4053-8431-e12fa57229c6\") "
Jul 30 17:43:19.651075 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.651041    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e8d615-a180-4053-8431-e12fa57229c6-plugins-config-volume\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.653052 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.652977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f" (OuterVolumeSpecName: "kube-api-access-m9t4f") pod "93e8d615-a180-4053-8431-e12fa57229c6" (UID: "93e8d615-a180-4053-8431-e12fa57229c6"). InnerVolumeSpecName "kube-api-access-m9t4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:19.751435 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:19.751404    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9t4f\" (UniqueName: \"kubernetes.io/projected/93e8d615-a180-4053-8431-e12fa57229c6-kube-api-access-m9t4f\") on node \"ip-10-0-132-133.ec2.internal\" DevicePath \"\""
Jul 30 17:43:20.374736 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.374699    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"
Jul 30 17:43:20.374736 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.374730    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj" event={"ID":"93e8d615-a180-4053-8431-e12fa57229c6","Type":"ContainerDied","Data":"d40f0ad5c93f0b7b92dfed3e16481d74d9ba2651d3c50d817c9e96a0573a735b"}
Jul 30 17:43:20.375280 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.374794    2561 scope.go:117] "RemoveContainer" containerID="14ae4217c588a6b2b8626102ab866cb1a0ef1fc84e8befc139a061fd243f1c2f"
Jul 30 17:43:20.376245 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.376185    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"
Jul 30 17:43:20.376313 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.376182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh" event={"ID":"d9b0103e-aed1-4ce0-8c7c-234f7563100f","Type":"ContainerDied","Data":"d9d3b089628d9ccde40b22b8cb36a116bf9e18030fb5e27d8929f31c46cff506"}
Jul 30 17:43:20.384763 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.384746    2561 scope.go:117] "RemoveContainer" containerID="cc6c2c07675185d7482e28dafef86a2787d8c8d33f31f07b55724d4d5541eed8"
Jul 30 17:43:20.400243 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.400221    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:43:20.404235 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.404213    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-27deb9-859c75c5bb-dfksh"]
Jul 30 17:43:20.414174 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.414149    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:43:20.418674 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:20.418620    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-27deb9-76ff677ffb-r59wj"]
Jul 30 17:43:21.340734 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:21.340699    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e8d615-a180-4053-8431-e12fa57229c6" path="/var/lib/kubelet/pods/93e8d615-a180-4053-8431-e12fa57229c6/volumes"
Jul 30 17:43:21.341078 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:21.341066    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" path="/var/lib/kubelet/pods/d9b0103e-aed1-4ce0-8c7c-234f7563100f/volumes"
Jul 30 17:43:22.326712 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:22.326682    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rclnf"]
Jul 30 17:43:26.542922 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:26.542893    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77dbcddb95-8w79v_8a07c376-a8b6-41b0-891c-6584a1339cd6/authorino/0.log"
Jul 30 17:43:31.946563 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:31.946524    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bf2lt_41a88185-9b73-4109-ab1c-34da262e55de/manager/0.log"
Jul 30 17:43:32.074639 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:32.074611    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mcgv4_361f5299-762b-4cf5-8144-dd09e4c68bbd/manager/0.log"
Jul 30 17:43:32.343852 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:32.343829    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-q9m9w_637a960b-22d5-47bf-af57-92486c952828/server/0.log"
Jul 30 17:43:32.464396 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:32.464356    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tn6pp_de85fd46-7924-43fc-8952-f02d11366e71/manager/0.log"
Jul 30 17:43:32.584769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:32.584698    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-49vbm_1f8fd8bf-afc1-4c0f-ba0a-caf458418db8/manager/0.log"
Jul 30 17:43:32.592077 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:32.592052    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-49vbm_1f8fd8bf-afc1-4c0f-ba0a-caf458418db8/copy-manifests/0.log"
Jul 30 17:43:33.455853 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:33.455819    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757195-kldvm_5ae1a241-397a-4bf1-8558-1b6b5d524fb1/cleanup/0.log"
Jul 30 17:43:35.033889 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.033855    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77dbcddb95-8w79v_8a07c376-a8b6-41b0-891c-6584a1339cd6/authorino/0.log"
Jul 30 17:43:35.157802 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.157768    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wtfzz_6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6/manager/0.log"
Jul 30 17:43:35.268423 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.268393    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-tz7gs_1c45526a-02fa-4223-8153-f89ce872f074/manager/0.log"
Jul 30 17:43:35.505732 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.505701    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p6s5l_d0a491a4-d01f-4c57-9fee-efce71c04b57/registry-server/0.log"
Jul 30 17:43:35.629197 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.629161    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rpbrq_c8d3bcb5-21a4-4299-863a-a406d25ccc4a/manager/0.log"
Jul 30 17:43:35.740535 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.740510    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rclnf_919c174f-f968-4953-b5c6-c168c512f43d/limitador/0.log"
Jul 30 17:43:35.869095 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:35.869040    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kbdbx_76f335cd-4c08-46de-afcc-7bd95e2dfa69/manager/0.log"
Jul 30 17:43:36.460643 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:36.460614    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-5jrdw_f01341b6-8c3a-4278-90d0-e59fb8b0da85/kube-auth-proxy/0.log"
Jul 30 17:43:37.067879 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:37.067844    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6748bdbfdc-jrch2_c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6/router/0.log"
Jul 30 17:43:37.764979 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:37.764949    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7kvvd_ea31c6b4-eedd-44e0-a589-4ba77f822f7e/main/0.log"
Jul 30 17:43:37.995111 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:37.995076    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8pq7j_031f1e58-2320-4655-9610-073f9344bf0e/main/0.log"
Jul 30 17:43:43.918151 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918115    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pt5b5/must-gather-6ghqw"]
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918462    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" containerName="revoke-keys"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918473    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" containerName="revoke-keys"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918521    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" containerName="payload-pre-processing"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918530    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" containerName="payload-pre-processing"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918539    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e8d615-a180-4053-8431-e12fa57229c6" containerName="payload-pre-processing"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918550    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e8d615-a180-4053-8431-e12fa57229c6" containerName="payload-pre-processing"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918560    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" containerName="payload-processing"
Jul 30 17:43:43.918594 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918568    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" containerName="payload-processing"
Jul 30 17:43:43.918870 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918628    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e8d615-a180-4053-8431-e12fa57229c6" containerName="payload-pre-processing"
Jul 30 17:43:43.918870 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918636    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dda4bbe-fb6c-4833-b759-efe57cfdbc46" containerName="payload-pre-processing"
Jul 30 17:43:43.918870 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918644    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9b0103e-aed1-4ce0-8c7c-234f7563100f" containerName="payload-processing"
Jul 30 17:43:43.918870 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.918649    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2823cac3-dd6b-42c9-8cc1-1c2c876fd0c6" containerName="revoke-keys"
Jul 30 17:43:43.921799 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.921782    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:43.924776 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.924753    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pt5b5\"/\"default-dockercfg-mvv78\""
Jul 30 17:43:43.925023 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.925002    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pt5b5\"/\"kube-root-ca.crt\""
Jul 30 17:43:43.925121 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.925012    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pt5b5\"/\"openshift-service-ca.crt\""
Jul 30 17:43:43.937690 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:43.937664    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/must-gather-6ghqw"]
Jul 30 17:43:44.063013 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.062971    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f76846e3-c6be-4877-81d7-5c4ca803b3fc-must-gather-output\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.063210 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.063025    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvvq\" (UniqueName: \"kubernetes.io/projected/f76846e3-c6be-4877-81d7-5c4ca803b3fc-kube-api-access-fmvvq\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.164295 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.164259    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f76846e3-c6be-4877-81d7-5c4ca803b3fc-must-gather-output\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.164525 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.164302    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvvq\" (UniqueName: \"kubernetes.io/projected/f76846e3-c6be-4877-81d7-5c4ca803b3fc-kube-api-access-fmvvq\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.164642 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.164621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f76846e3-c6be-4877-81d7-5c4ca803b3fc-must-gather-output\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.172605 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.172539    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvvq\" (UniqueName: \"kubernetes.io/projected/f76846e3-c6be-4877-81d7-5c4ca803b3fc-kube-api-access-fmvvq\") pod \"must-gather-6ghqw\" (UID: \"f76846e3-c6be-4877-81d7-5c4ca803b3fc\") " pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.231324 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.231281    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/must-gather-6ghqw"
Jul 30 17:43:44.392838 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.392809    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/must-gather-6ghqw"]
Jul 30 17:43:44.394708 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:43:44.394682    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf76846e3_c6be_4877_81d7_5c4ca803b3fc.slice/crio-ade5669bb28e632e63cbf1aa10123f46733c30a3d8d89d73da7a428d88b4b382 WatchSource:0}: Error finding container ade5669bb28e632e63cbf1aa10123f46733c30a3d8d89d73da7a428d88b4b382: Status 404 returned error can't find the container with id ade5669bb28e632e63cbf1aa10123f46733c30a3d8d89d73da7a428d88b4b382
Jul 30 17:43:44.396593 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.396575    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:43:44.493264 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:44.493166    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/must-gather-6ghqw" event={"ID":"f76846e3-c6be-4877-81d7-5c4ca803b3fc","Type":"ContainerStarted","Data":"ade5669bb28e632e63cbf1aa10123f46733c30a3d8d89d73da7a428d88b4b382"}
Jul 30 17:43:45.500972 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:45.500935    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/must-gather-6ghqw" event={"ID":"f76846e3-c6be-4877-81d7-5c4ca803b3fc","Type":"ContainerStarted","Data":"ed12ef100589ba6f72b23ec52d07ff284aafc6e89afbd712a6b88789d95a401d"}
Jul 30 17:43:46.511703 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:46.511642    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/must-gather-6ghqw" event={"ID":"f76846e3-c6be-4877-81d7-5c4ca803b3fc","Type":"ContainerStarted","Data":"f8a076fb1f1bd9a100f935fb9230993bf64c9d7dce413ddf9366e973d618b9b1"}
Jul 30 17:43:46.528268 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:46.528221    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pt5b5/must-gather-6ghqw" podStartSLOduration=2.535616235 podStartE2EDuration="3.52820566s" totalImagesPullingTime="992.589425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:43:46.527139042 +0000 UTC m=+4055.784600463" watchObservedRunningTime="2026-07-30 17:43:46.52820566 +0000 UTC m=+4055.785667127"
Jul 30 17:43:47.004270 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:47.004240    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mkxgs_741ecd44-3b6f-4429-9900-e03e8d8710b9/global-pull-secret-syncer/0.log"
Jul 30 17:43:47.114888 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:47.114855    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r6x55_1a71b369-4cc8-4eaa-aa69-bced7def4dce/konnectivity-agent/0.log"
Jul 30 17:43:47.195042 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:47.195008    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-133.ec2.internal_ceb15f8e15e94f7eca74166940c49314/haproxy/0.log"
Jul 30 17:43:52.557542 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.557509    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77dbcddb95-8w79v_8a07c376-a8b6-41b0-891c-6584a1339cd6/authorino/0.log"
Jul 30 17:43:52.606639 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.606607    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wtfzz_6b228f4b-2cd2-45e0-9e60-fcddd3b0dfd6/manager/0.log"
Jul 30 17:43:52.650618 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.650585    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-tz7gs_1c45526a-02fa-4223-8153-f89ce872f074/manager/0.log"
Jul 30 17:43:52.764068 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.764034    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p6s5l_d0a491a4-d01f-4c57-9fee-efce71c04b57/registry-server/0.log"
Jul 30 17:43:52.915513 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.915404    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rpbrq_c8d3bcb5-21a4-4299-863a-a406d25ccc4a/manager/0.log"
Jul 30 17:43:52.942781 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:52.942606    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rclnf_919c174f-f968-4953-b5c6-c168c512f43d/limitador/0.log"
Jul 30 17:43:53.058506 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:53.058436    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kbdbx_76f335cd-4c08-46de-afcc-7bd95e2dfa69/manager/0.log"
Jul 30 17:43:55.292383 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:55.292344    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-plgb2_7f66f702-5be0-43fd-b57c-1a1dd8219483/node-exporter/0.log"
Jul 30 17:43:55.321346 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:55.321304    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-plgb2_7f66f702-5be0-43fd-b57c-1a1dd8219483/kube-rbac-proxy/0.log"
Jul 30 17:43:55.346990 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:55.346956    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-plgb2_7f66f702-5be0-43fd-b57c-1a1dd8219483/init-textfile/0.log"
Jul 30 17:43:57.244434 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.244392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"]
Jul 30 17:43:57.250340 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.250308    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.256721 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.256686    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"]
Jul 30 17:43:57.409268 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.409214    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z888g\" (UniqueName: \"kubernetes.io/projected/ff753480-12bb-4311-94c5-b30aa2a7481f-kube-api-access-z888g\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.409467 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.409322    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-sys\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.409467 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.409346    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-proc\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.409467 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.409366    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-podres\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.409467 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.409420    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-lib-modules\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510497 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510396    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z888g\" (UniqueName: \"kubernetes.io/projected/ff753480-12bb-4311-94c5-b30aa2a7481f-kube-api-access-z888g\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510497 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-sys\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-sys\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-proc\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510651    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-podres\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-lib-modules\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.510752 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510714    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-proc\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.511046 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510775    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-podres\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.511046 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.510848    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff753480-12bb-4311-94c5-b30aa2a7481f-lib-modules\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.520318 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.520269    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z888g\" (UniqueName: \"kubernetes.io/projected/ff753480-12bb-4311-94c5-b30aa2a7481f-kube-api-access-z888g\") pod \"perf-node-gather-daemonset-s2dmc\" (UID: \"ff753480-12bb-4311-94c5-b30aa2a7481f\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.567895 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.567858    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:57.764050 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.763955    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"]
Jul 30 17:43:57.766512 ip-10-0-132-133 kubenswrapper[2561]: W0730 17:43:57.766449    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff753480_12bb_4311_94c5_b30aa2a7481f.slice/crio-bf96542d45f90249d21e057bc76c5d9853d603f8df9eaecbefe8b54b7527cfa0 WatchSource:0}: Error finding container bf96542d45f90249d21e057bc76c5d9853d603f8df9eaecbefe8b54b7527cfa0: Status 404 returned error can't find the container with id bf96542d45f90249d21e057bc76c5d9853d603f8df9eaecbefe8b54b7527cfa0
Jul 30 17:43:57.903882 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.903838    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/1.log"
Jul 30 17:43:57.914814 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:57.914784    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kskrz_9f5c0566-e219-43bd-8ecf-de5a4e7eed78/console-operator/2.log"
Jul 30 17:43:58.606298 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:58.606256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc" event={"ID":"ff753480-12bb-4311-94c5-b30aa2a7481f","Type":"ContainerStarted","Data":"2c3abdeae812aa0a9f6f46671fbf1bf00b141cbb9eb681bb693078312326527d"}
Jul 30 17:43:58.606721 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:58.606314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc" event={"ID":"ff753480-12bb-4311-94c5-b30aa2a7481f","Type":"ContainerStarted","Data":"bf96542d45f90249d21e057bc76c5d9853d603f8df9eaecbefe8b54b7527cfa0"}
Jul 30 17:43:58.606721 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:58.606434    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:43:58.623588 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:58.623531    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc" podStartSLOduration=1.623508056 podStartE2EDuration="1.623508056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:43:58.621604673 +0000 UTC m=+4067.879066125" watchObservedRunningTime="2026-07-30 17:43:58.623508056 +0000 UTC m=+4067.880969477"
Jul 30 17:43:58.985997 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:43:58.985906    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-xhz55_9b77609b-cf37-4900-a46a-14f76238bf45/volume-data-source-validator/0.log"
Jul 30 17:44:00.006322 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:00.006294    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99jng_bc28418d-5fec-4aac-a992-e57d9832495b/dns/0.log"
Jul 30 17:44:00.028084 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:00.028050    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99jng_bc28418d-5fec-4aac-a992-e57d9832495b/kube-rbac-proxy/0.log"
Jul 30 17:44:00.176285 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:00.176256    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mdp4w_f214bfb0-038b-4596-8951-c85042453a54/dns-node-resolver/0.log"
Jul 30 17:44:00.795783 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:00.795695    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m5bt4_177be080-0a05-4ad1-bb8e-6b162983cf44/node-ca/0.log"
Jul 30 17:44:01.927787 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:01.927754    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-5jrdw_f01341b6-8c3a-4278-90d0-e59fb8b0da85/kube-auth-proxy/0.log"
Jul 30 17:44:02.117057 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:02.117025    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6748bdbfdc-jrch2_c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6/router/0.log"
Jul 30 17:44:02.967462 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:02.967430    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nl5h6_86204387-75d5-43b6-be45-9f6b12167f39/serve-healthcheck-canary/0.log"
Jul 30 17:44:03.572843 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:03.572813    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djcbt_862368d2-77e5-425b-a497-1caf9e2cb930/kube-rbac-proxy/0.log"
Jul 30 17:44:03.594132 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:03.594100    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djcbt_862368d2-77e5-425b-a497-1caf9e2cb930/exporter/0.log"
Jul 30 17:44:03.616720 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:03.616691    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djcbt_862368d2-77e5-425b-a497-1caf9e2cb930/extractor/0.log"
Jul 30 17:44:04.626407 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:04.626378    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-s2dmc"
Jul 30 17:44:05.912845 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:05.912815    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bf2lt_41a88185-9b73-4109-ab1c-34da262e55de/manager/0.log"
Jul 30 17:44:06.348998 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:06.348964    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mcgv4_361f5299-762b-4cf5-8144-dd09e4c68bbd/manager/0.log"
Jul 30 17:44:06.618933 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:06.618894    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-q9m9w_637a960b-22d5-47bf-af57-92486c952828/server/0.log"
Jul 30 17:44:06.752544 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:06.752502    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tn6pp_de85fd46-7924-43fc-8952-f02d11366e71/manager/0.log"
Jul 30 17:44:06.779087 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:06.779062    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-49vbm_1f8fd8bf-afc1-4c0f-ba0a-caf458418db8/manager/0.log"
Jul 30 17:44:06.801732 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:06.801707    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-49vbm_1f8fd8bf-afc1-4c0f-ba0a-caf458418db8/copy-manifests/0.log"
Jul 30 17:44:08.120769 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:08.120741    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-5jrdw_f01341b6-8c3a-4278-90d0-e59fb8b0da85/kube-auth-proxy/0.log"
Jul 30 17:44:08.252808 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:08.252779    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6748bdbfdc-jrch2_c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6/router/0.log"
Jul 30 17:44:09.814204 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:09.814125    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-5jrdw_f01341b6-8c3a-4278-90d0-e59fb8b0da85/kube-auth-proxy/0.log"
Jul 30 17:44:09.957848 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:09.957815    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6748bdbfdc-jrch2_c4ec96c4-2282-4a6c-acfa-2a2111cf2ce6/router/0.log"
Jul 30 17:44:13.822942 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:13.822900    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c_22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09/kube-storage-version-migrator-operator/1.log"
Jul 30 17:44:13.824875 ip-10-0-132-133 kubenswrapper[2561]: I0730 17:44:13.824847    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s6g2c_22b967ae-4cc3-4cff-bcd0-3a0e9ccc6a09/kube-storage-version-migrator-operator/0.log"