Jul 23 18:58:55.176670 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 18:58:55.176681 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 18:58:55.176690 ip-10-0-135-11 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 18:58:55.176998 ip-10-0-135-11 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 18:59:05.426043 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 18:59:05.426065 ip-10-0-135-11 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b6f1657b3a2443d896b015ceaa0331d7 --
Jul 23 19:00:55.939262 ip-10-0-135-11 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.260097    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265525    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265540    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265543    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265547    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265550    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265553    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265556    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.545489 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265559    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.325235 ip-10-0-135-11 systemd[1]: Started Kubernetes Kubelet.
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265561    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265564    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265567    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265570    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265572    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265598    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265602    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265606    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265609    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265612    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265615    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265618    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265621    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265623    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265626    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265629    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265632    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265634    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265637    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.559791 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265640    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265644    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265646    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265651    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265655    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265659    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265662    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265665    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265668    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265671    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265673    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265676    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265679    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265682    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265685    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265689    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265692    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265695    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.561220 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265698    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265702    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265705    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265709    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265712    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265715    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265718    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265721    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265723    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265727    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265731    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265733    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265736    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265739    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265741    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265744    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265747    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265750    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265753    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.562324 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265755    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265758    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265760    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265764    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265768    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265770    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265773    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265776    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265778    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265781    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265783    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265785    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265788    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265790    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265793    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265796    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265798    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265801    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265803    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265806    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.564405 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265809    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265811    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265814    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265816    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265819    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265821    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265824    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265826    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265829    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265832    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265835    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265837    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265840    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265842    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.265845    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266560    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266566    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266569    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266572    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266575    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.566373 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266578    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266581    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266584    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266587    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266589    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266592    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266594    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266597    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266599    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266602    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266605    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266608    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266611    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266613    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266618    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266622    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266625    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266628    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266631    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.568256 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266633    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266636    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266638    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266641    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266643    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266647    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266651    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266654    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266657    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266659    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266678    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266682    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266685    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266689    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266693    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266696    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266708    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266712    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266715    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.570418 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266719    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266722    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266725    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266727    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266730    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266733    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266735    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266739    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266741    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266744    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266746    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266749    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266752    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266757    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266760    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266764    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266766    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266769    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266772    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266774    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.571583 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266776    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266779    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266782    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266785    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266787    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266790    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266792    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266794    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266797    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266800    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266802    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266805    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266807    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266811    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266814    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266817    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266820    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266823    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266826    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.572292 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266829    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266832    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266835    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266838    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266840    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266843    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266848    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266851    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266853    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266856    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266858    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266861    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266863    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266866    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266868    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266871    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.266874    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267260    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267266    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267269    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.572888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267272    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267275    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267278    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267281    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267284    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267288    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267292    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267296    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267299    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267302    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267304    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267307    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267310    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267313    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267316    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267318    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267321    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267325    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267329    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.573524 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267332    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267335    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267338    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267340    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267343    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267346    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267349    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267351    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267354    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267356    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267372    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267376    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267380    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267383    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267386    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267389    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267392    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267395    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267398    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267400    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.574131 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267403    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267406    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267409    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267411    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267414    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267416    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267419    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267422    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267425    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267428    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267430    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267434    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267437    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267439    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267442    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267447    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267450    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267453    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.574753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267457    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267460    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267462    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267465    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267468    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267470    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267473    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267477    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267481    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267485    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267487    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267490    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267494    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267496    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267499    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267501    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267504    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267506    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267509    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267511    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.575804 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267514    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267517    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267519    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267522    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267526    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267528    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267531    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267533    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267536    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267538    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267541    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267543    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267546    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267548    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267551    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267553    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267556    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.267559    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267620    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267627    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267633    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 19:00:56.577189 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267638    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267647    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267651    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267655    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267659    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267662    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267665    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267668    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267672    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267675    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267677    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267680    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267683    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267687    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267690    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267695    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267698    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267701    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267704    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267707    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267711    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267714    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267717    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 19:00:56.577760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267720    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267723    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267726    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267729    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267732    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267735    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267739    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267742    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267744    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267749    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267752    2563 flags.go:64] FLAG: --enable-server="true"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267755    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267759    2563 flags.go:64] FLAG: --event-burst="100"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267762    2563 flags.go:64] FLAG: --event-qps="50"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267765    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267769    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267771    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267775    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267778    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267780    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267783    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267786    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267789    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267793    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267797    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 19:00:56.578416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267800    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267803    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267805    2563 flags.go:64] FLAG: --feature-gates=""
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267809    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267812    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267815    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267818    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267821    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267824    2563 flags.go:64] FLAG: --help="false"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267827    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267830    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267833    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267836    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267839    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267842    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267845    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267850    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267853    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267855    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267858    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267861    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267864    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267867    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267870    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 19:00:56.579135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267872    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267875    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267878    2563 flags.go:64] FLAG: --lock-file=""
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267881    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267884    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267887    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267892    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267897    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267899    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267902    2563 flags.go:64] FLAG: --logging-format="text"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267905    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267908    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267911    2563 flags.go:64] FLAG: --manifest-url=""
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267914    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267917    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267920    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267924    2563 flags.go:64] FLAG: --max-pods="110"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267927    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267930    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267933    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267936    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267938    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267941    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267945    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267969    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 19:00:56.579792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267973    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267976    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267979    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267982    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267987    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267990    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267992    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267995    2563 flags.go:64] FLAG: --port="10250"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.267999    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268001    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035aa92f7df23ba76"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268005    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268008    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268010    2563 flags.go:64] FLAG: --register-node="true"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268013    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268017    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268024    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268027    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268030    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268034    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268037    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268040    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268043    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268045    2563 flags.go:64] FLAG: --runonce="false"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268048    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268051    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 19:00:56.580872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268054    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268057    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268060    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268063    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268066    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268069    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268073    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268076    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268079    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268082    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268085    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268087    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268090    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268095    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268098    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268101    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268104    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268107    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268110    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268113    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268116    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268119    2563 flags.go:64] FLAG: --v="2"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268124    2563 flags.go:64] FLAG: --version="false"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268128    2563 flags.go:64] FLAG: --vmodule=""
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268132    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 19:00:56.581600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268135    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268244    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268248    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268251    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268255    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268257    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268260    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268265    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268269    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268272    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268275    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268278    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268281    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268285    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268288    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268290    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268293    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268295    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268298    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.783202 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268300    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268303    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268306    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268309    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268311    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268314    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268316    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268319    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268321    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268324    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268328    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268330    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268334    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268336    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268339    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268341    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268344    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268346    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268349    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.784540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268351    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268354    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268356    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268359    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268361    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268364    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268366    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268370    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268373    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268375    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268378    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268380    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268383    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268385    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268388    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268390    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268393    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268395    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268398    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268400    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268403    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.785604 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268405    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268408    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268412    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268415    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268417    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268424    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268427    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268430    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268433    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268435    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268438    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268440    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268443    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268448    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268450    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268453    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268455    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268459    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.786885 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268461    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268464    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268466    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268469    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268472    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268475    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268477    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268481    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268485    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268488    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268491    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268494    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268497    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268500    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268504    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268508    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268511    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268513    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268516    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.787891 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268519    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268522    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.268524    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.268557    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.276645    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.276661    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.276682    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.276690    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276746    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276751    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276754    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276758    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276760    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276763    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276766    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.789140 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276769    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276772    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276774    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276777    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276779    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276782    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276785    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276787    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276790    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276792    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276795    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276798    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276801    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276804    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276806    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276809    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276811    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276815    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276818    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.790004 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276820    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276825    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276828    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276831    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276834    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276837    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276839    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276842    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276845    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276847    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276850    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276853    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276855    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276858    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276861    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276864    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276867    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276869    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276872    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.791072 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276875    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276877    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276880    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276882    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276885    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276887    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276890    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276892    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276895    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276897    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276900    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276906    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276911    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276914    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276916    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276919    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276922    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276925    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276927    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276929    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276932    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.792152 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276935    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276937    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276940    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276942    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276945    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276947    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276966    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276970    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276976    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276979    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276982    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276985    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276988    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276990    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276993    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276995    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.276999    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277003    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277006    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.793207 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277009    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277011    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277014    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277021    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277025    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277028    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277031    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277034    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277037    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277040    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277042    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277045    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277048    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277050    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.794195 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.277055    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277162    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277166    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277169    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277172    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277175    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277178    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277181    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277184    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277187    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277190    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277193    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277195    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277198    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277201    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277203    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277206    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277208    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277211    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277214    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:00:56.795092 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277216    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277220    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277223    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277227    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277230    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277232    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277235    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277237    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277240    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277242    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277245    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277248    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277250    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277253    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277255    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277258    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277260    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277263    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277265    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277268    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:00:56.796118 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277270    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277273    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277275    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277277    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277280    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277284    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277288    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277291    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277294    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277297    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277299    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277302    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277305    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277309    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277312    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277316    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277318    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277326    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277329    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:00:56.797224 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277332    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277334    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277337    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277339    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277342    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277344    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277347    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277350    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277353    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277356    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277360    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277363    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277366    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277369    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277371    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277373    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277376    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277379    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277381    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277383    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:00:56.798261 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277386    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277389    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277391    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277393    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277396    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277399    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277403    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277407    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277412    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277415    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277418    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277421    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277424    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277427    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277430    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277433    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277435    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277438    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277441    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:00:56.799330 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277443    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:56.277446    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.277451    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.277546    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.280092    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.281209    2563 server.go:1019] "Starting client certificate rotation"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.281305    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.281334    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.283493    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.284896    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.294334    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.296292    2563 log.go:25] "Validated CRI v1 image API"
Jul 23 19:00:56.800386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.297354    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 19:00:56.801410 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.299316    2563 fs.go:135] Filesystem UUIDs: map[3a5c10b1-1fb9-4010-b431-eea1522399e9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c2b35ea5-fe5b-4d3e-b01a-95c37f1344f5:/dev/nvme0n1p3]
Jul 23 19:00:56.801410 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.299331    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.304280    2563 manager.go:216] Machine: {Timestamp:2026-07-23 19:00:56.302910658 +0000 UTC m=+0.262355585 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100460 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23db22e49ce80110f0f41605ea649c SystemUUID:ec23db22-e49c-e801-10f0-f41605ea649c BootID:b6f1657b-3a24-43d8-96b0-15ceaa0331d7 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:f2:fb:70:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:f2:fb:70:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:16:0c:08:a0:1e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.304394    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.304480    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.307073    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.307094    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.307666    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.307675    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.307682    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.308232    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.309399    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.312559    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313361    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313493    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313526    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313539    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313552    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 19:00:56.801881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313562    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313571    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313902    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.313927    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.315679    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.317780    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.317817    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320059    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320086    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320099    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320109    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320125    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320133    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320144    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320152    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320163    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320172    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320184    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320192    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320740    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.320753    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.324244    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.324361    2563 server.go:1268] "Started kubelet"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.324324    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.324425    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.324445    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.324462    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.324578    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.328562    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.329737    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.330900    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.332767    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-11.ec2.internal.18c50063e28328b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-11.ec2.internal,UID:ip-10-0-135-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-11.ec2.internal,},FirstTimestamp:2026-07-23 19:00:56.324278456 +0000 UTC m=+0.283723379,LastTimestamp:2026-07-23 19:00:56.324278456 +0000 UTC m=+0.283723379,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-11.ec2.internal,}"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.334423    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.334430    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.334494    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.334920    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335003    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335022    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335227    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335236    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335863    2563 factory.go:55] Registering systemd factory
Jul 23 19:00:56.804425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.335879    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.336182    2563 factory.go:153] Registering CRI-O factory
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.336195    2563 factory.go:223] Registration of the crio container factory successfully
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.336259    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.336283    2563 factory.go:103] Registering Raw factory
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.336297    2563 manager.go:1196] Started watching for new ooms in manager
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.337346    2563 manager.go:349] Starting recovery of all containers
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.338585    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.338685    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.339611    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-25nxf"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.348559    2563 manager.go:354] Recovery completed
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.350061    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-25nxf"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.353514    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.355566    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.355589    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.355597    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356146    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356153    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356166    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356217    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356909    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356936    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.356965    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.357704    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-11.ec2.internal.18c50063e460b97b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-11.ec2.internal,UID:ip-10-0-135-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-11.ec2.internal,},FirstTimestamp:2026-07-23 19:00:56.355576187 +0000 UTC m=+0.315021112,LastTimestamp:2026-07-23 19:00:56.355576187 +0000 UTC m=+0.315021112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-11.ec2.internal,}"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.358173    2563 policy_none.go:49] "None policy: Start"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.358186    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.358198    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.360112    2563 policy_none.go:47] "Start"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427495    2563 manager.go:337] "Starting Device Plugin manager"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.427529    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427540    2563 server.go:87] "Starting device plugin registration server"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427735    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427745    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427872    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427981    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.427991    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.428638    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.428689    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.457146    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.458444    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 19:00:56.806720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.458476    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.458495    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.458537    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.461105    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.528193    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.528992    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.529019    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.529036    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.529059    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.536420    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.536438    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-11.ec2.internal\": node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.552458    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.559492    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"]
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.559548    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.560382    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.560408    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.560417    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.562449    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.562604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.562640    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563248    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563273    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563249    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563307    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563319    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.563283    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.565275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.565301    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.565996    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.566022    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.566031    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.599583    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-11.ec2.internal\" not found" node="ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.603891    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-11.ec2.internal\" not found" node="ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.635792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.981673 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.635814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.635830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0cf1e41fbd8c7e6ff0f79ac186a0f32c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"0cf1e41fbd8c7e6ff0f79ac186a0f32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.653409    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0cf1e41fbd8c7e6ff0f79ac186a0f32c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"0cf1e41fbd8c7e6ff0f79ac186a0f32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0cf1e41fbd8c7e6ff0f79ac186a0f32c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"0cf1e41fbd8c7e6ff0f79ac186a0f32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736640    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.736651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ebea944135483b8635748f86c6e0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"7ebea944135483b8635748f86c6e0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.753627    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.854483    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.902632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:56.906149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:56.982914 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:56.954999    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:57.206947 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:57.055434    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:57.206947 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:57.155880    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found"
Jul 23 19:00:57.206947 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:57.165566    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.291495    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.303296    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.304798    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.321596    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.337731    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.337836    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.337869    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.371063    2563 apiserver.go:52] "Watching apiserver"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.387017    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.387333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-xmvgp","kube-system/global-pull-secret-syncer-fwvxh","kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal","openshift-dns/node-resolver-rcdmw","openshift-image-registry/node-ca-gm8mt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal","openshift-multus/multus-4x4ss","openshift-multus/multus-additional-cni-plugins-dbldv","openshift-multus/network-metrics-daemon-9s5gk","kube-system/konnectivity-agent-wf4ks","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w","openshift-cluster-node-tuning-operator/tuned-b7zj7","openshift-network-diagnostics/network-check-target-4sbt5","openshift-network-operator/iptables-alerter-lzw8n"]
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.389679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.391659    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.392409    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.392414    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.393912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jhpsj\""
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.394219    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.394691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.394797    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.394984    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-system-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-hostroot\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-etc-kubernetes\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.509463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-kubelet\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9prsq\" (UniqueName: \"kubernetes.io/projected/cbc7db65-d860-49cc-9d8c-8273119557e4-kube-api-access-9prsq\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-bin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-multus\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.395980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-cnibin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.397346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-os-release\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-k8s-cni-cncf-io\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-conf-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-cni-binary-copy\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-socket-dir-parent\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-daemon-config\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-multus-certs\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-netns\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.398826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.399692    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.399711    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.399733    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.399778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.400500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jgngt\""
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.401872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.402778    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.510308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.404017    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.406050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.406107    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.406758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nq9pp\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.407864    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.408039    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.408063    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.408146    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.408543    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.409243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fj7gq\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.409454    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hqxvj\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.409622    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.409642    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.410241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.410387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.410449    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.411631    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 18:55:57 +0000 UTC" deadline="2028-01-04 16:29:30.053960257 +0000 UTC"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.411674    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12717h28m31.642289207s"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.412583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.412660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.412661    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rnrlk\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.412866    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.413064    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.413272    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.413934    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.415287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.415716    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-265hr\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.416265    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.416499    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.417178    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.417615    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.417645    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.417770    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.417901    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p6cd8\""
Jul 23 19:00:58.511098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.418827    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.419195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t7ljg\""
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.419492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.429913    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.437918    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l6cx4"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.449450    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l6cx4"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.493028    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.498969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-k8s-cni-cncf-io\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.498993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-systemd-units\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-socket-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-os-release\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7p6\" (UniqueName: \"kubernetes.io/projected/e0f03efa-faba-4a1a-8541-652b8324483f-kube-api-access-px7p6\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-k8s-cni-cncf-io\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-socket-dir-parent\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-host\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-netns\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-conf\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-netns\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8e1e141-7e22-45a1-93a2-deebc035b718-ovn-node-metrics-cert\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-netns\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chcs\" (UniqueName: \"kubernetes.io/projected/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-kube-api-access-8chcs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-socket-dir-parent\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-ovn\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-multus\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-cnibin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499332    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-multus\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-kubelet\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-config\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-cnibin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-kubelet-config\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7aa3a18f-3a3c-4295-a156-25bacab288bb-iptables-alerter-script\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-tuned\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-script-lib\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.512798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-system-cni-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-hostroot\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bda1177-dc5b-41fd-849d-f590926553b7-hosts-file\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-dbus\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-device-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-hostroot\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpv58\" (UniqueName: \"kubernetes.io/projected/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kube-api-access-vpv58\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-modprobe-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-var-lib-kubelet\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-host\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbmh\" (UniqueName: \"kubernetes.io/projected/da07f2de-6f49-426d-964a-0f12a8da4c3a-kube-api-access-txbmh\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bda1177-dc5b-41fd-849d-f590926553b7-tmp-dir\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-cnibin\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.513406 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysconfig\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.499991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-conf-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-multus-certs\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e0b68d4-6d40-4b89-9e98-34232914551f-agent-certs\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b68d4-6d40-4b89-9e98-34232914551f-konnectivity-ca\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-run-multus-certs\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-conf-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-netd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhr6f\" (UniqueName: \"kubernetes.io/projected/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-kube-api-access-dhr6f\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-cni-binary-copy\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-systemd\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-sys\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-bin\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7aa3a18f-3a3c-4295-a156-25bacab288bb-host-slash\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-system-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.513956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-lib-modules\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-system-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h85kd\" (UniqueName: \"kubernetes.io/projected/8bda1177-dc5b-41fd-849d-f590926553b7-kube-api-access-h85kd\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-slash\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-env-overrides\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-serviceca\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-cni-binary-copy\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-kubelet\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-kubelet\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2vw5\" (UniqueName: \"kubernetes.io/projected/e8e1e141-7e22-45a1-93a2-deebc035b718-kube-api-access-f2vw5\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-kubernetes\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-os-release\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-daemon-config\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-node-log\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.514777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-log-socket\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-tmp\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-systemd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-os-release\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-registration-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-cni-dir\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.500989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-etc-kubernetes\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-etc-kubernetes\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-var-lib-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-sys-fs\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9prsq\" (UniqueName: \"kubernetes.io/projected/cbc7db65-d860-49cc-9d8c-8273119557e4-kube-api-access-9prsq\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-run\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-etc-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxwsf\" (UniqueName: \"kubernetes.io/projected/7aa3a18f-3a3c-4295-a156-25bacab288bb-kube-api-access-zxwsf\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-bin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.515368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbc7db65-d860-49cc-9d8c-8273119557e4-multus-daemon-config\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.501248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbc7db65-d860-49cc-9d8c-8273119557e4-host-var-lib-cni-bin\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.505851    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.505865    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.505874    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.505922    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.005905694 +0000 UTC m=+1.908491749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.506994    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 19:00:58.516018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.510336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9prsq\" (UniqueName: \"kubernetes.io/projected/cbc7db65-d860-49cc-9d8c-8273119557e4-kube-api-access-9prsq\") pod \"multus-4x4ss\" (UID: \"cbc7db65-d860-49cc-9d8c-8273119557e4\") " pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-systemd-units\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-socket-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-os-release\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-systemd-units\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7p6\" (UniqueName: \"kubernetes.io/projected/e0f03efa-faba-4a1a-8541-652b8324483f-kube-api-access-px7p6\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-socket-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-host\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-conf\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-host\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-os-release\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-netns\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8e1e141-7e22-45a1-93a2-deebc035b718-ovn-node-metrics-cert\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chcs\" (UniqueName: \"kubernetes.io/projected/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-kube-api-access-8chcs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-netns\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-ovn\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.728018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-ovn\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-conf\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-kubelet\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-config\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-kubelet-config\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7aa3a18f-3a3c-4295-a156-25bacab288bb-iptables-alerter-script\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-tuned\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.602658    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-kubelet-config\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-kubelet\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.602734    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.102703048 +0000 UTC m=+2.005289105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-script-lib\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.729583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-system-cni-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bda1177-dc5b-41fd-849d-f590926553b7-hosts-file\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-dbus\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-device-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv58\" (UniqueName: \"kubernetes.io/projected/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kube-api-access-vpv58\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-modprobe-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.602989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-var-lib-kubelet\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e0324-af23-4a58-9836-35b49754f300-dbus\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-host\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8bda1177-dc5b-41fd-849d-f590926553b7-hosts-file\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbmh\" (UniqueName: \"kubernetes.io/projected/da07f2de-6f49-426d-964a-0f12a8da4c3a-kube-api-access-txbmh\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bda1177-dc5b-41fd-849d-f590926553b7-tmp-dir\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-cnibin\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysconfig\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-device-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.730565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-config\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e0b68d4-6d40-4b89-9e98-34232914551f-agent-certs\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-modprobe-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b68d4-6d40-4b89-9e98-34232914551f-konnectivity-ca\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-host\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-var-lib-kubelet\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysctl-d\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-cnibin\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-netd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhr6f\" (UniqueName: \"kubernetes.io/projected/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-kube-api-access-dhr6f\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bda1177-dc5b-41fd-849d-f590926553b7-tmp-dir\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-systemd\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-sys\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7aa3a18f-3a3c-4295-a156-25bacab288bb-iptables-alerter-script\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.731657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-bin\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-sysconfig\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603698    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-bin\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-sys\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7aa3a18f-3a3c-4295-a156-25bacab288bb-host-slash\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-systemd\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-lib-modules\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h85kd\" (UniqueName: \"kubernetes.io/projected/8bda1177-dc5b-41fd-849d-f590926553b7-kube-api-access-h85kd\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-cni-netd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7aa3a18f-3a3c-4295-a156-25bacab288bb-host-slash\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-slash\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-lib-modules\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-env-overrides\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:58.733096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-serviceca\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.603982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2vw5\" (UniqueName: \"kubernetes.io/projected/e8e1e141-7e22-45a1-93a2-deebc035b718-kube-api-access-f2vw5\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-kubernetes\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-node-log\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-log-socket\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b68d4-6d40-4b89-9e98-34232914551f-konnectivity-ca\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-tmp\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-systemd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-registration-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-kubernetes\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-var-lib-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-host-slash\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-sys-fs\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-run\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.733743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-etc-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-env-overrides\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwsf\" (UniqueName: \"kubernetes.io/projected/7aa3a18f-3a3c-4295-a156-25bacab288bb-kube-api-access-zxwsf\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-serviceca\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.604430    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-sys-fs\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.604476    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.104462625 +0000 UTC m=+2.007048685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-run-systemd\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-registration-dir\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-var-lib-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-node-log\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-log-socket\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da07f2de-6f49-426d-964a-0f12a8da4c3a-run\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.604962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8e1e141-7e22-45a1-93a2-deebc035b718-etc-openvswitch\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.605230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8e1e141-7e22-45a1-93a2-deebc035b718-ovn-node-metrics-cert\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.605283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0f03efa-faba-4a1a-8541-652b8324483f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.734478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.605619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0f03efa-faba-4a1a-8541-652b8324483f-system-cni-dir\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.606212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e0b68d4-6d40-4b89-9e98-34232914551f-agent-certs\") pod \"konnectivity-agent-wf4ks\" (UID: \"9e0b68d4-6d40-4b89-9e98-34232914551f\") " pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.606523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8e1e141-7e22-45a1-93a2-deebc035b718-ovnkube-script-lib\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.606617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-tmp\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.611798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da07f2de-6f49-426d-964a-0f12a8da4c3a-etc-tuned\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.613592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7p6\" (UniqueName: \"kubernetes.io/projected/e0f03efa-faba-4a1a-8541-652b8324483f-kube-api-access-px7p6\") pod \"multus-additional-cni-plugins-dbldv\" (UID: \"e0f03efa-faba-4a1a-8541-652b8324483f\") " pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.617730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chcs\" (UniqueName: \"kubernetes.io/projected/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-kube-api-access-8chcs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.623420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwsf\" (UniqueName: \"kubernetes.io/projected/7aa3a18f-3a3c-4295-a156-25bacab288bb-kube-api-access-zxwsf\") pod \"iptables-alerter-lzw8n\" (UID: \"7aa3a18f-3a3c-4295-a156-25bacab288bb\") " pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.623465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhr6f\" (UniqueName: \"kubernetes.io/projected/9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7-kube-api-access-dhr6f\") pod \"node-ca-gm8mt\" (UID: \"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7\") " pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.623545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpv58\" (UniqueName: \"kubernetes.io/projected/aa780290-aeb6-4c99-8dfb-fdefb7fc59eb-kube-api-access-vpv58\") pod \"aws-ebs-csi-driver-node-ndh5w\" (UID: \"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.625049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbmh\" (UniqueName: \"kubernetes.io/projected/da07f2de-6f49-426d-964a-0f12a8da4c3a-kube-api-access-txbmh\") pod \"tuned-b7zj7\" (UID: \"da07f2de-6f49-426d-964a-0f12a8da4c3a\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.625342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2vw5\" (UniqueName: \"kubernetes.io/projected/e8e1e141-7e22-45a1-93a2-deebc035b718-kube-api-access-f2vw5\") pod \"ovnkube-node-xmvgp\" (UID: \"e8e1e141-7e22-45a1-93a2-deebc035b718\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.625350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h85kd\" (UniqueName: \"kubernetes.io/projected/8bda1177-dc5b-41fd-849d-f590926553b7-kube-api-access-h85kd\") pod \"node-resolver-rcdmw\" (UID: \"8bda1177-dc5b-41fd-849d-f590926553b7\") " pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.681985    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf1e41fbd8c7e6ff0f79ac186a0f32c.slice/crio-6812fca9276c506762e5c3cd451e210180e4c35a7e8f054d1084db5eab4de0f0 WatchSource:0}: Error finding container 6812fca9276c506762e5c3cd451e210180e4c35a7e8f054d1084db5eab4de0f0: Status 404 returned error can't find the container with id 6812fca9276c506762e5c3cd451e210180e4c35a7e8f054d1084db5eab4de0f0
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.684276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebea944135483b8635748f86c6e0810.slice/crio-e16068affa317b05d773cf339dff7fd7e1e432033dbc131bd89cb65fde787af2 WatchSource:0}: Error finding container e16068affa317b05d773cf339dff7fd7e1e432033dbc131bd89cb65fde787af2: Status 404 returned error can't find the container with id e16068affa317b05d773cf339dff7fd7e1e432033dbc131bd89cb65fde787af2
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.684571    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.689088    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.720510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4x4ss"
Jul 23 19:00:58.735148 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.730753    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc7db65_d860_49cc_9d8c_8273119557e4.slice/crio-6036589183ba85691593981b261b981a109372484411e7649e623cfa139f1088 WatchSource:0}: Error finding container 6036589183ba85691593981b261b981a109372484411e7649e623cfa139f1088: Status 404 returned error can't find the container with id 6036589183ba85691593981b261b981a109372484411e7649e623cfa139f1088
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.742797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lzw8n"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.748636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rcdmw"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.750383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa3a18f_3a3c_4295_a156_25bacab288bb.slice/crio-8e4c1340035d6a1e0a71aa77905bef79db07cf9e0ab716765ce010084b008d32 WatchSource:0}: Error finding container 8e4c1340035d6a1e0a71aa77905bef79db07cf9e0ab716765ce010084b008d32: Status 404 returned error can't find the container with id 8e4c1340035d6a1e0a71aa77905bef79db07cf9e0ab716765ce010084b008d32
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.756699    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bda1177_dc5b_41fd_849d_f590926553b7.slice/crio-d1e1e4afa1c793680facfbada3cd1377ba7f94eb2900fc13883b039062baab6f WatchSource:0}: Error finding container d1e1e4afa1c793680facfbada3cd1377ba7f94eb2900fc13883b039062baab6f: Status 404 returned error can't find the container with id d1e1e4afa1c793680facfbada3cd1377ba7f94eb2900fc13883b039062baab6f
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.772570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gm8mt"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.780323    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbb4dde_6efd_4b8c_98c1_d2ec0124d5f7.slice/crio-ac3096136adfc3f6c22094036dd149ad5335ae6686bcb334fbb18e75e3132094 WatchSource:0}: Error finding container ac3096136adfc3f6c22094036dd149ad5335ae6686bcb334fbb18e75e3132094: Status 404 returned error can't find the container with id ac3096136adfc3f6c22094036dd149ad5335ae6686bcb334fbb18e75e3132094
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.794593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dbldv"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.802204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f03efa_faba_4a1a_8541_652b8324483f.slice/crio-cb12fab5a4410b6b981c6cd05dbab00b5f2603dc88b3312d2e7e7d3b37c1f9d7 WatchSource:0}: Error finding container cb12fab5a4410b6b981c6cd05dbab00b5f2603dc88b3312d2e7e7d3b37c1f9d7: Status 404 returned error can't find the container with id cb12fab5a4410b6b981c6cd05dbab00b5f2603dc88b3312d2e7e7d3b37c1f9d7
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.819672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.827011    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e1e141_7e22_45a1_93a2_deebc035b718.slice/crio-bcb05dfc957063ce2a5fc243b7ac5b86bfa71a62bec63c92250f73a279573aa3 WatchSource:0}: Error finding container bcb05dfc957063ce2a5fc243b7ac5b86bfa71a62bec63c92250f73a279573aa3: Status 404 returned error can't find the container with id bcb05dfc957063ce2a5fc243b7ac5b86bfa71a62bec63c92250f73a279573aa3
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.847452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.855080    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0b68d4_6d40_4b89_9e98_34232914551f.slice/crio-f21b6f71cb69570f533b44df60140f6dbb908ac7e0bb9d8f7194619e132981e1 WatchSource:0}: Error finding container f21b6f71cb69570f533b44df60140f6dbb908ac7e0bb9d8f7194619e132981e1: Status 404 returned error can't find the container with id f21b6f71cb69570f533b44df60140f6dbb908ac7e0bb9d8f7194619e132981e1
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.871978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.877991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:58.878082    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.879544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa780290_aeb6_4c99_8dfb_fdefb7fc59eb.slice/crio-52bfca22dfbb2be3d8ae550a7acfaf4ea3959ed2fe8efb5a7e907b8f84f4bcd0 WatchSource:0}: Error finding container 52bfca22dfbb2be3d8ae550a7acfaf4ea3959ed2fe8efb5a7e907b8f84f4bcd0: Status 404 returned error can't find the container with id 52bfca22dfbb2be3d8ae550a7acfaf4ea3959ed2fe8efb5a7e907b8f84f4bcd0
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:00:58.886207    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda07f2de_6f49_426d_964a_0f12a8da4c3a.slice/crio-0c767aba6cb6383ec10f8c80eef26194b6adf5eb62c4f89a16b109c632f642de WatchSource:0}: Error finding container 0c767aba6cb6383ec10f8c80eef26194b6adf5eb62c4f89a16b109c632f642de: Status 404 returned error can't find the container with id 0c767aba6cb6383ec10f8c80eef26194b6adf5eb62c4f89a16b109c632f642de
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.888135    2563 kuberuntime_manager.go:1459] "Unhandled Error" err="container tuned start failed in pod tuned-b7zj7_openshift-cluster-node-tuning-operator(da07f2de-6f49-426d-964a-0f12a8da4c3a): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 23 19:00:58.962647 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:58.888161    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7" podUID="da07f2de-6f49-426d-964a-0f12a8da4c3a"
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.007566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.007721    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.007740    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.007753    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.007796    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:00.007783531 +0000 UTC m=+2.910369588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.108421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.108476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.108609    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.108668    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:01:00.108650039 +0000 UTC m=+3.011236098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.109022    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:00:59.154099 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.109064    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:00.109051011 +0000 UTC m=+3.011637067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:00:59.462434 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.452967    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:55:58 +0000 UTC" deadline="2028-02-19 06:26:32.803605212 +0000 UTC"
Jul 23 19:00:59.462434 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.453023    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13811h25m33.350586311s"
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.533835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4x4ss" event={"ID":"cbc7db65-d860-49cc-9d8c-8273119557e4","Type":"ContainerStarted","Data":"6036589183ba85691593981b261b981a109372484411e7649e623cfa139f1088"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.538066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"7ebea944135483b8635748f86c6e0810","Type":"ContainerStarted","Data":"e16068affa317b05d773cf339dff7fd7e1e432033dbc131bd89cb65fde787af2"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.546378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" event={"ID":"0cf1e41fbd8c7e6ff0f79ac186a0f32c","Type":"ContainerStarted","Data":"6812fca9276c506762e5c3cd451e210180e4c35a7e8f054d1084db5eab4de0f0"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.549524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w" event={"ID":"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb","Type":"ContainerStarted","Data":"52bfca22dfbb2be3d8ae550a7acfaf4ea3959ed2fe8efb5a7e907b8f84f4bcd0"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.555286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerStarted","Data":"cb12fab5a4410b6b981c6cd05dbab00b5f2603dc88b3312d2e7e7d3b37c1f9d7"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.560869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gm8mt" event={"ID":"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7","Type":"ContainerStarted","Data":"ac3096136adfc3f6c22094036dd149ad5335ae6686bcb334fbb18e75e3132094"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.586246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lzw8n" event={"ID":"7aa3a18f-3a3c-4295-a156-25bacab288bb","Type":"ContainerStarted","Data":"8e4c1340035d6a1e0a71aa77905bef79db07cf9e0ab716765ce010084b008d32"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.590513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7" event={"ID":"da07f2de-6f49-426d-964a-0f12a8da4c3a","Type":"ContainerStarted","Data":"0c767aba6cb6383ec10f8c80eef26194b6adf5eb62c4f89a16b109c632f642de"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.593886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wf4ks" event={"ID":"9e0b68d4-6d40-4b89-9e98-34232914551f","Type":"ContainerStarted","Data":"f21b6f71cb69570f533b44df60140f6dbb908ac7e0bb9d8f7194619e132981e1"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:00:59.594903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7" podUID="da07f2de-6f49-426d-964a-0f12a8da4c3a"
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.597475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"bcb05dfc957063ce2a5fc243b7ac5b86bfa71a62bec63c92250f73a279573aa3"}
Jul 23 19:00:59.608315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.602645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcdmw" event={"ID":"8bda1177-dc5b-41fd-849d-f590926553b7","Type":"ContainerStarted","Data":"d1e1e4afa1c793680facfbada3cd1377ba7f94eb2900fc13883b039062baab6f"}
Jul 23 19:00:59.827139 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:00:59.827003    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:01:00.017545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.017512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:00.017744 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.017727    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:01:00.017802 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.017753    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:01:00.017802 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.017767    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:00.017932 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.017842    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:02.01780775 +0000 UTC m=+4.920393820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:00.118721 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.118636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:00.118721 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.118708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:00.119182 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.118890    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:00.119182 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.118929    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:00.119182 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.118953    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:01:02.118932601 +0000 UTC m=+5.021518661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:00.119182 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.118993    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:02.118974297 +0000 UTC m=+5.021560353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:00.127615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.127592    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:01:00.455005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.454892    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:55:58 +0000 UTC" deadline="2028-03-22 11:03:26.270973894 +0000 UTC"
Jul 23 19:01:00.455005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.454952    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14584h2m25.816026309s"
Jul 23 19:01:00.516480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.516448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:00.516641 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.516581    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:00.517007 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.516989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:00.517113 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.517094    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:00.517175 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:00.517162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:00.517247 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:00.517230    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:02.035274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.035025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:02.035274 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.035178    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:01:02.035274 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.035201    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:01:02.035274 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.035213    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:02.035274 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.035270    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:06.03525308 +0000 UTC m=+8.937839154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:02.135829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.135781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:02.135987 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.135880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:02.136055 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.136000    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:02.136055 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.136050    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:06.136036667 +0000 UTC m=+9.038622723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:02.136518 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.136493    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:02.136619 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.136557    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:01:06.136541351 +0000 UTC m=+9.039127431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:02.516317 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.516219    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:02.516317 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.516259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:02.516539 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.516381    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:02.516714 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:02.516687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:02.516848 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.516790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:02.516910 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:02.516882    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:04.515872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:04.515841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:04.515872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:04.515879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:04.516370 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:04.515876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:04.516370 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:04.515940    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:04.516370 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:04.516006    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:04.516370 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:04.516084    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:06.072951 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.072869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:06.073387 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.073073    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:01:06.073387 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.073095    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:01:06.073387 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.073108    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:06.073387 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.073166    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:14.073147636 +0000 UTC m=+16.975733694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:06.173738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.173700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:06.173901 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.173787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:06.173959 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.173942    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:06.174016 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.174006    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:01:14.173986479 +0000 UTC m=+17.076572537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:06.174481 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.174393    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:06.174481 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.174449    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:14.174434276 +0000 UTC m=+17.077020334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:06.516511 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.516428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:06.516669 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.516561    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:06.516877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.516848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:06.517026 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.516958    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:06.517026 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:06.517006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:06.517149 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:06.517080    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:08.516358 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:08.516316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:08.516782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:08.516333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:08.516782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:08.516452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:08.516782 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:08.516449    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:08.516782 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:08.516537    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:08.516782 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:08.516635    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:10.516027 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:10.515968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:10.516439 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:10.516088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:10.516439 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:10.516098    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:10.516439 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:10.516094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:10.516439 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:10.516199    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:10.516439 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:10.516297    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:12.516471 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:12.516438    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:12.516892 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:12.516527    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:12.516892 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:12.516595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:12.516892 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:12.516674    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:12.516892 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:12.516723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:12.516892 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:12.516784    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:14.134331 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.134294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:14.134781 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.134407    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:01:14.134781 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.134423    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:01:14.134781 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.134432    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:14.134781 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.134488    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:30.134467652 +0000 UTC m=+33.037053709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:14.235523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.235490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:14.235681 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.235563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:14.235681 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.235660    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:14.235771 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.235732    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:30.23571016 +0000 UTC m=+33.138296221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:14.235771 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.235664    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:14.235895 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.235831    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:01:30.235799756 +0000 UTC m=+33.138385819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:14.515764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.515735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:14.515764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.515765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:14.515994 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:14.515735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:14.515994 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.515869    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:14.515994 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.515941    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:14.516115 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:14.516028    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:16.516183 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:16.516149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:16.516641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:16.516149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:16.516641 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:16.516278    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:16.516641 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:16.516369    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:16.516641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:16.516149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:16.516641 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:16.516455    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:18.516636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.516464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:18.517338 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.516509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:18.517338 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:18.516727    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:18.517338 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:18.516784    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:18.517338 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.516518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:18.517338 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:18.516861    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:18.645599 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.645571    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="704ae57e070f96d56d6553efbcaf325f0cff5752e49e920366ec5ff3d25bda18" exitCode=0
Jul 23 19:01:18.645699 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.645662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"704ae57e070f96d56d6553efbcaf325f0cff5752e49e920366ec5ff3d25bda18"}
Jul 23 19:01:18.647413 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.647392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gm8mt" event={"ID":"9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7","Type":"ContainerStarted","Data":"90b008041a6e3b877b84043f7e2f576b37f53498792c96c31c96a7c09962daa8"}
Jul 23 19:01:18.650785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.650752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7" event={"ID":"da07f2de-6f49-426d-964a-0f12a8da4c3a","Type":"ContainerStarted","Data":"9a61e90c436f35008a9fc257f2f98a0997d89a6d32828a96215a2eef3c420d01"}
Jul 23 19:01:18.652915 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.652891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wf4ks" event={"ID":"9e0b68d4-6d40-4b89-9e98-34232914551f","Type":"ContainerStarted","Data":"6f96fc788c2390a79b63fb7cad2019405fae79d15aaf2663d4b3116d3d68805d"}
Jul 23 19:01:18.656331 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.656310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"701e40c8b5dff87193618f8991abb6ba8bfdb1ec3bb3819d49475d61bf4c706c"}
Jul 23 19:01:18.656443 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.656385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"ec00bab8f8b4288a0cd8488febc38d665c1fde4caf43c69feb2899299457764d"}
Jul 23 19:01:18.656443 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.656439    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"eca27775a2f1057bb67dbb3c6be33ee9efa83a804ce1b92cf36483c65d89e31e"}
Jul 23 19:01:18.656554 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.656488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"d9d369b518cc7afd5066fbf0afbc979dc31e2b7efc4e5c0620dd817e4ad6bc36"}
Jul 23 19:01:18.656554 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.656536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"bb2dcf707455c6062a3b16578980f767b9f14c109587f36e831774e84379d313"}
Jul 23 19:01:18.657728 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.657706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rcdmw" event={"ID":"8bda1177-dc5b-41fd-849d-f590926553b7","Type":"ContainerStarted","Data":"88413ef472c85bbe97fefc1335ebc5a246f9bc9b672168d8762322264795c255"}
Jul 23 19:01:18.659319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.659293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4x4ss" event={"ID":"cbc7db65-d860-49cc-9d8c-8273119557e4","Type":"ContainerStarted","Data":"1a085b39775ca39abb790cb11dc96d7df5f70c41bc834a0876eda40bb13aaf0e"}
Jul 23 19:01:18.660930 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.660906    2563 generic.go:356] "Generic (PLEG): container finished" podID="7ebea944135483b8635748f86c6e0810" containerID="076f19fabfe4eb0776a63d8bd4c20c52a19d2171a9e49835486029a89264ed97" exitCode=0
Jul 23 19:01:18.661027 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.660944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"7ebea944135483b8635748f86c6e0810","Type":"ContainerDied","Data":"076f19fabfe4eb0776a63d8bd4c20c52a19d2171a9e49835486029a89264ed97"}
Jul 23 19:01:18.662551 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.662532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" event={"ID":"0cf1e41fbd8c7e6ff0f79ac186a0f32c","Type":"ContainerStarted","Data":"17bc7688408ab7e94b6f0be8de76e660309dee3b2c896761fe551cf82c13855c"}
Jul 23 19:01:18.664191 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.664160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w" event={"ID":"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb","Type":"ContainerStarted","Data":"3a952ac66995c020e3f84f66b58ac15538384cf909bd347c2b16c3c9abbb58be"}
Jul 23 19:01:18.714101 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.714048    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" podStartSLOduration=20.714032788 podStartE2EDuration="20.714032788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.713594493 +0000 UTC m=+21.616180571" watchObservedRunningTime="2026-07-23 19:01:18.714032788 +0000 UTC m=+21.616618944"
Jul 23 19:01:18.730050 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.729938    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gm8mt" podStartSLOduration=2.427487864 podStartE2EDuration="21.729918597s" totalImagesPullingTime="19.302430733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.729665915 +0000 UTC m=+21.632251995" watchObservedRunningTime="2026-07-23 19:01:18.729918597 +0000 UTC m=+21.632504677"
Jul 23 19:01:18.745655 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.745599    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wf4ks" podStartSLOduration=2.517756426 podStartE2EDuration="21.745578377s" totalImagesPullingTime="19.227821951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.745028219 +0000 UTC m=+21.647614307" watchObservedRunningTime="2026-07-23 19:01:18.745578377 +0000 UTC m=+21.648164457"
Jul 23 19:01:18.783959 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.783912    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b7zj7" podStartSLOduration=2.512566965 podStartE2EDuration="21.783895265s" totalImagesPullingTime="19.2713283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-23 19:01:18.764852774 +0000 UTC m=+21.667438851" watchObservedRunningTime="2026-07-23 19:01:18.783895265 +0000 UTC m=+21.686481343"
Jul 23 19:01:18.805547 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:18.805510    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4x4ss" podStartSLOduration=2.452561525 podStartE2EDuration="21.805497409s" totalImagesPullingTime="19.352935884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.805277232 +0000 UTC m=+21.707863309" watchObservedRunningTime="2026-07-23 19:01:18.805497409 +0000 UTC m=+21.708083484"
Jul 23 19:01:19.668034 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.667773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lzw8n" event={"ID":"7aa3a18f-3a3c-4295-a156-25bacab288bb","Type":"ContainerStarted","Data":"b7c48ea7df238f9126f2c1064570e9fe5cfc5773feba87e7e8551ee4a3d75ac2"}
Jul 23 19:01:19.671183 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.671151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"b6295bd88a2a7097e05c0f72cdcc17a19882d3ded02044dc0c19423f45831cfa"}
Jul 23 19:01:19.673114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.673088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"7ebea944135483b8635748f86c6e0810","Type":"ContainerStarted","Data":"370369de79d41fd6edc9c0f0779dacae1dca4b4a1164b711b62d9fa0c5388dfd"}
Jul 23 19:01:19.684972 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.684862    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lzw8n" podStartSLOduration=5.247253651 podStartE2EDuration="22.684846088s" totalImagesPullingTime="17.437592437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:19.684671095 +0000 UTC m=+22.587257192" watchObservedRunningTime="2026-07-23 19:01:19.684846088 +0000 UTC m=+22.587432167"
Jul 23 19:01:19.685192 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.685165    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rcdmw" podStartSLOduration=3.358404824 podStartE2EDuration="22.685157361s" totalImagesPullingTime="19.326752537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.823657369 +0000 UTC m=+21.726243557" watchObservedRunningTime="2026-07-23 19:01:19.685157361 +0000 UTC m=+22.587743439"
Jul 23 19:01:19.700616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:19.700553    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" podStartSLOduration=21.700536742 podStartE2EDuration="21.700536742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:19.699951993 +0000 UTC m=+22.602538071" watchObservedRunningTime="2026-07-23 19:01:19.700536742 +0000 UTC m=+22.603122821"
Jul 23 19:01:20.013579 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.013531    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 19:01:20.494377 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.494243    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T19:01:20.013559658Z","UUID":"05f906b5-cf33-49b1-b781-96768001b596","Handler":null,"Name":"","Endpoint":""}
Jul 23 19:01:20.495747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.495726    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 19:01:20.495893 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.495769    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 19:01:20.516039 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.516018    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:20.516159 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:20.516104    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:20.516159 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.516116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:20.516236 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:20.516194    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:20.516236 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.516110    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:20.516304 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:20.516285    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:20.677754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:20.677719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w" event={"ID":"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb","Type":"ContainerStarted","Data":"5612418325c22a8f96951e4c3aa24087c9f2ee888332d04a332f6b7272c4edce"}
Jul 23 19:01:21.330405 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:21.330377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:01:21.330903 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:21.330884    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:01:21.683637 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:21.683556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"6a79e33c4bad0689f29eb3a64c5ae6a8da12a70721953acd9e777ab424c4f82e"}
Jul 23 19:01:21.684053 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:21.683646    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:01:21.684279 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:21.684265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wf4ks"
Jul 23 19:01:22.516237 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:22.516203    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:22.516237 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:22.516237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:22.516437 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:22.516237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:22.516437 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:22.516338    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:22.516530 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:22.516459    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:22.516530 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:22.516520    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:23.691570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.691333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" event={"ID":"e8e1e141-7e22-45a1-93a2-deebc035b718","Type":"ContainerStarted","Data":"cc4040b10e38157eb74f7cc614a41d45de11423a88e24f81bf46b4625c072c0c"}
Jul 23 19:01:23.692052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.691609    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:23.692052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.691657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:23.693580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.693543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w" event={"ID":"aa780290-aeb6-4c99-8dfb-fdefb7fc59eb","Type":"ContainerStarted","Data":"c22aac0688cda302cb34d16b590b776a8d231b0e8d8eec57b9e5adbacd415e32"}
Jul 23 19:01:23.695344 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.695315    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="e825c38620b5124d8b69db590045c7e36e61fd96b3238f10af0af4f5040ba755" exitCode=0
Jul 23 19:01:23.695453 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.695389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"e825c38620b5124d8b69db590045c7e36e61fd96b3238f10af0af4f5040ba755"}
Jul 23 19:01:23.707159 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.707139    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:23.770217 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.770171    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp" podStartSLOduration=7.512438057 podStartE2EDuration="26.77015944s" totalImagesPullingTime="19.257721383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:23.734932136 +0000 UTC m=+26.637518211" watchObservedRunningTime="2026-07-23 19:01:23.77015944 +0000 UTC m=+26.672745518"
Jul 23 19:01:23.821862 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:23.821825    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ndh5w" podStartSLOduration=3.123185742 podStartE2EDuration="26.821800544s" totalImagesPullingTime="23.698614802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:23.792963755 +0000 UTC m=+26.695549832" watchObservedRunningTime="2026-07-23 19:01:23.821800544 +0000 UTC m=+26.724386622"
Jul 23 19:01:24.515956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.515929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:24.516076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.516055    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:24.516076 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:24.516057    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:24.516155 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.516074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:24.516155 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:24.516136    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:24.516219 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:24.516192    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:24.700013 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.699935    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="d3a399f0a87fd5e345db3083d31f37ec15075b8c5844748b7aa3bd559b887c44" exitCode=0
Jul 23 19:01:24.700641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.700015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"d3a399f0a87fd5e345db3083d31f37ec15075b8c5844748b7aa3bd559b887c44"}
Jul 23 19:01:24.700776 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.700686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:24.715782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:24.715762    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:25.712459 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.712310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fwvxh"]
Jul 23 19:01:25.712948 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.712523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:25.712948 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:25.712630    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:25.715563 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.715530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9s5gk"]
Jul 23 19:01:25.715676 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.715630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:25.715758 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:25.715735    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:25.719770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.719748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sbt5"]
Jul 23 19:01:25.719881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:25.719840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:25.719953 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:25.719928    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:27.518492 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:27.518464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:27.519001 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:27.518565    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:27.519001 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:27.518573    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:27.519001 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:27.518591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:27.519001 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:27.518664    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:27.519001 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:27.518738    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:27.709464 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:27.709433    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="7072f17893c98b18a549d5c02db8ca7f33a6b77b5bc8e3d5c718bfcb2e4d9507" exitCode=0
Jul 23 19:01:27.709605 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:27.709474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"7072f17893c98b18a549d5c02db8ca7f33a6b77b5bc8e3d5c718bfcb2e4d9507"}
Jul 23 19:01:29.515788 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:29.515754    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:29.515788 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:29.515754    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:29.516450 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:29.515879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:29.516450 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:29.515881    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sbt5" podUID="dcf73f6a-37e7-4997-a658-354c29cbabd0"
Jul 23 19:01:29.516450 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:29.515971    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5gk" podUID="d05c0c74-6f50-4551-a0c2-6c8da9b2eace"
Jul 23 19:01:29.516450 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:29.516052    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fwvxh" podUID="4a3e0324-af23-4a58-9836-35b49754f300"
Jul 23 19:01:30.160478 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.160442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:30.160660 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.160632    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:01:30.160660 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.160654    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:01:30.160767 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.160667    2563 projected.go:196] Error preparing data for projected volume kube-api-access-qhn9b for pod openshift-network-diagnostics/network-check-target-4sbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:30.160767 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.160735    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b podName:dcf73f6a-37e7-4997-a658-354c29cbabd0 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.160713702 +0000 UTC m=+65.063299762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qhn9b" (UniqueName: "kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b") pod "network-check-target-4sbt5" (UID: "dcf73f6a-37e7-4997-a658-354c29cbabd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:01:30.260854 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.260827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:30.261011 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.260880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:30.261011 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.260968    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:30.261011 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.260983    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:30.261146 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.261016    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret podName:4a3e0324-af23-4a58-9836-35b49754f300 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.261002665 +0000 UTC m=+65.163588721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret") pod "global-pull-secret-syncer-fwvxh" (UID: "4a3e0324-af23-4a58-9836-35b49754f300") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:01:30.261146 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:30.261039    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs podName:d05c0c74-6f50-4551-a0c2-6c8da9b2eace nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.261022583 +0000 UTC m=+65.163608656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs") pod "network-metrics-daemon-9s5gk" (UID: "d05c0c74-6f50-4551-a0c2-6c8da9b2eace") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:01:30.886184 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.886114    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeReady"
Jul 23 19:01:30.886562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.886242    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 19:01:30.942264 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.942233    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:01:30.969548 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.969520    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dxxx6"]
Jul 23 19:01:30.969707 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.969656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:30.975929 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.975739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 19:01:30.975929 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.975742    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 19:01:30.975929 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.975840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 19:01:30.976159 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.975932    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nv7r4\""
Jul 23 19:01:30.985419 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.985401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:01:30.985539 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.985490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:30.986159 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.986131    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xg9q2"]
Jul 23 19:01:30.997707 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.997676    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 19:01:30.998007 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.997961    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 19:01:30.998007 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.997996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kngdw\""
Jul 23 19:01:30.998167 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.998041    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 19:01:30.998167 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:30.998092    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 19:01:31.003394 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.003372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dxxx6"]
Jul 23 19:01:31.003488 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.003476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.008253 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.007293    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fgf46\""
Jul 23 19:01:31.008253 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.007426    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 19:01:31.008253 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.007516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 19:01:31.013587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.013565    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xg9q2"]
Jul 23 19:01:31.067637 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067605    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-cert\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bj7c\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.067794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvcfc\" (UniqueName: \"kubernetes.io/projected/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-kube-api-access-qvcfc\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.068095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.068095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.067977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-cert\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.168752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvqgj\" (UniqueName: \"kubernetes.io/projected/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-kube-api-access-fvqgj\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bj7c\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvcfc\" (UniqueName: \"kubernetes.io/projected/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-kube-api-access-qvcfc\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-config-volume\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-tmp-dir\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.168980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.168944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.169301 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.169265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.169444 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.169405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.169571 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.169503    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:31.169571 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.169519    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:31.169674 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.169585    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.669565146 +0000 UTC m=+34.572151209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:31.170092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.170068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.173971 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.173951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.174069 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.173980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-cert\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.174069 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.173953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.181285 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.181244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvcfc\" (UniqueName: \"kubernetes.io/projected/44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c-kube-api-access-qvcfc\") pod \"ingress-canary-dxxx6\" (UID: \"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c\") " pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.182459 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.182439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bj7c\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.182865 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.182840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.270093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.270254 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.270194    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:31.270308 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.270272    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.770253067 +0000 UTC m=+34.672839122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:31.270308 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvqgj\" (UniqueName: \"kubernetes.io/projected/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-kube-api-access-fvqgj\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.270398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-config-volume\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.270398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-tmp-dir\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.270696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-tmp-dir\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.270933 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.270910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-config-volume\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.281989 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.281945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvqgj\" (UniqueName: \"kubernetes.io/projected/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-kube-api-access-fvqgj\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.295846 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.295823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dxxx6"
Jul 23 19:01:31.515692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.515628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:01:31.515692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.515645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:01:31.515936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.515779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:01:31.519885 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.519665    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 19:01:31.519885 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.519803    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 19:01:31.520075 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.519900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5ztj\""
Jul 23 19:01:31.520185 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.520124    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 19:01:31.520425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.520407    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8qs8\""
Jul 23 19:01:31.521042 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.520970    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 19:01:31.674006 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.673968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:31.674173 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.674120    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:31.674173 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.674144    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:31.674261 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.674211    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.674192892 +0000 UTC m=+35.576778951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:31.775346 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:31.775265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:31.775501 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.775408    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:31.775501 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:31.775478    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.775460028 +0000 UTC m=+35.678046098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:32.040487 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.040464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dxxx6"]
Jul 23 19:01:32.136393 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:01:32.136354    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d7b1f5_0ee1_48af_8ea1_17059eb3ba0c.slice/crio-cd490f3e1c0b9b669c0819243c5e3bfe25666f86cc73db499aa86b981600475a WatchSource:0}: Error finding container cd490f3e1c0b9b669c0819243c5e3bfe25666f86cc73db499aa86b981600475a: Status 404 returned error can't find the container with id cd490f3e1c0b9b669c0819243c5e3bfe25666f86cc73db499aa86b981600475a
Jul 23 19:01:32.683629 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.683550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:32.683797 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:32.683727    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:32.683797 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:32.683747    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:32.683930 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:32.683830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.683793845 +0000 UTC m=+37.586379902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:32.721387 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.721350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dxxx6" event={"ID":"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c","Type":"ContainerStarted","Data":"cd490f3e1c0b9b669c0819243c5e3bfe25666f86cc73db499aa86b981600475a"}
Jul 23 19:01:32.724420 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.724368    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="78d3f08124d6174ba22c9d13c8be5579df521841f0f65f38d9f823a8ee2bef58" exitCode=0
Jul 23 19:01:32.724420 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.724402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"78d3f08124d6174ba22c9d13c8be5579df521841f0f65f38d9f823a8ee2bef58"}
Jul 23 19:01:32.784410 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:32.784378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:32.784863 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:32.784549    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:32.784863 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:32.784613    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.784594445 +0000 UTC m=+37.687180520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:33.280528 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.280491    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"]
Jul 23 19:01:33.294901 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.294870    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"]
Jul 23 19:01:33.295024 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.294957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"
Jul 23 19:01:33.299551 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.299514    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xm4dq\""
Jul 23 19:01:33.389616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.389571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh69q\" (UniqueName: \"kubernetes.io/projected/c3bb2259-798d-43df-ac58-3a3c468910be-kube-api-access-sh69q\") pod \"network-check-source-7bc5b6bf85-kzdhn\" (UID: \"c3bb2259-798d-43df-ac58-3a3c468910be\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"
Jul 23 19:01:33.490297 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.490251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh69q\" (UniqueName: \"kubernetes.io/projected/c3bb2259-798d-43df-ac58-3a3c468910be-kube-api-access-sh69q\") pod \"network-check-source-7bc5b6bf85-kzdhn\" (UID: \"c3bb2259-798d-43df-ac58-3a3c468910be\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"
Jul 23 19:01:33.499601 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.499580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh69q\" (UniqueName: \"kubernetes.io/projected/c3bb2259-798d-43df-ac58-3a3c468910be-kube-api-access-sh69q\") pod \"network-check-source-7bc5b6bf85-kzdhn\" (UID: \"c3bb2259-798d-43df-ac58-3a3c468910be\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"
Jul 23 19:01:33.606105 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.606035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"
Jul 23 19:01:33.732643 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.731855    2563 generic.go:356] "Generic (PLEG): container finished" podID="e0f03efa-faba-4a1a-8541-652b8324483f" containerID="518471bc53bbd1aa1cde0f89a8014d4c56007b39cc99453c1a73727bcec5f497" exitCode=0
Jul 23 19:01:33.732643 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.731934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerDied","Data":"518471bc53bbd1aa1cde0f89a8014d4c56007b39cc99453c1a73727bcec5f497"}
Jul 23 19:01:33.745875 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:33.745852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn"]
Jul 23 19:01:33.748827 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:01:33.748790    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bb2259_798d_43df_ac58_3a3c468910be.slice/crio-d6c49f115853c1c9ce9a561b21390103a9ec6f2695e53bc8ce5d1d70192bc531 WatchSource:0}: Error finding container d6c49f115853c1c9ce9a561b21390103a9ec6f2695e53bc8ce5d1d70192bc531: Status 404 returned error can't find the container with id d6c49f115853c1c9ce9a561b21390103a9ec6f2695e53bc8ce5d1d70192bc531
Jul 23 19:01:34.699853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:34.699801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:34.700205 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:34.699966    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:34.700205 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:34.699983    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:34.700205 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:34.700042    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.700025059 +0000 UTC m=+41.602611115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:34.739206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:34.739167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbldv" event={"ID":"e0f03efa-faba-4a1a-8541-652b8324483f","Type":"ContainerStarted","Data":"dee8134553c434ffee6367e1789c53be0b6716bf977b7c310b0ce77ac1144383"}
Jul 23 19:01:34.740239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:34.740216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn" event={"ID":"c3bb2259-798d-43df-ac58-3a3c468910be","Type":"ContainerStarted","Data":"d6c49f115853c1c9ce9a561b21390103a9ec6f2695e53bc8ce5d1d70192bc531"}
Jul 23 19:01:34.765538 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:34.765502    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dbldv" podStartSLOduration=7.073836515 podStartE2EDuration="37.765488827s" totalImagesPullingTime="30.691652312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:34.764767188 +0000 UTC m=+37.667353288" watchObservedRunningTime="2026-07-23 19:01:34.765488827 +0000 UTC m=+37.668074904"
Jul 23 19:01:34.800516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:34.800491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:34.800607 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:34.800591    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:34.800657 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:34.800633    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.800622019 +0000 UTC m=+41.703208075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:35.744609 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:35.744571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dxxx6" event={"ID":"44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c","Type":"ContainerStarted","Data":"45d179acddd1708d771883c045376e92463a417ebc7d5f82324b05696da5d398"}
Jul 23 19:01:35.777251 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:35.777196    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dxxx6" podStartSLOduration=3.083643568 podStartE2EDuration="5.777174097s" totalImagesPullingTime="2.693530529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:35.776763492 +0000 UTC m=+38.679349572" watchObservedRunningTime="2026-07-23 19:01:35.777174097 +0000 UTC m=+38.679760176"
Jul 23 19:01:37.750619 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:37.750584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn" event={"ID":"c3bb2259-798d-43df-ac58-3a3c468910be","Type":"ContainerStarted","Data":"124f03c8aeeeb0f2003d948c942ed1bd990004c677ddc016997aa8ccbb92727d"}
Jul 23 19:01:38.730623 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:38.730585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:38.730792 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:38.730745    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:38.730792 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:38.730766    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:38.730885 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:38.730842    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.73080594 +0000 UTC m=+49.633391996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:38.831521 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:38.831490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:38.831877 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:38.831638    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:38.831877 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:38.831691    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.831675998 +0000 UTC m=+49.734262059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:39.053775 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.053730    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kzdhn" podStartSLOduration=2.245014189 podStartE2EDuration="6.053714015s" totalImagesPullingTime="3.808699826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:37.787319253 +0000 UTC m=+40.689905332" watchObservedRunningTime="2026-07-23 19:01:39.053714015 +0000 UTC m=+41.956300093"
Jul 23 19:01:39.054009 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.053995    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"]
Jul 23 19:01:39.087080 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.087057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"]
Jul 23 19:01:39.087190 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.087149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"
Jul 23 19:01:39.089832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.089800    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 23 19:01:39.090072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.090056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 23 19:01:39.091751 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.091734    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-s5m6p\""
Jul 23 19:01:39.234803 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.234770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jw6p\" (UniqueName: \"kubernetes.io/projected/6f9cb974-5a29-4c83-a121-1752f84ee609-kube-api-access-6jw6p\") pod \"migrator-7cc9c7ffd7-c9d8f\" (UID: \"6f9cb974-5a29-4c83-a121-1752f84ee609\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"
Jul 23 19:01:39.335263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.335190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jw6p\" (UniqueName: \"kubernetes.io/projected/6f9cb974-5a29-4c83-a121-1752f84ee609-kube-api-access-6jw6p\") pod \"migrator-7cc9c7ffd7-c9d8f\" (UID: \"6f9cb974-5a29-4c83-a121-1752f84ee609\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"
Jul 23 19:01:39.345707 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.345680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jw6p\" (UniqueName: \"kubernetes.io/projected/6f9cb974-5a29-4c83-a121-1752f84ee609-kube-api-access-6jw6p\") pod \"migrator-7cc9c7ffd7-c9d8f\" (UID: \"6f9cb974-5a29-4c83-a121-1752f84ee609\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"
Jul 23 19:01:39.395589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.395566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"
Jul 23 19:01:39.537859 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.537830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f"]
Jul 23 19:01:39.540940 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:01:39.540918    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9cb974_5a29_4c83_a121_1752f84ee609.slice/crio-4bd01f360937c1674fb8a0e6b0ff3ed7a04b0896365523c6c873166efc08ac1b WatchSource:0}: Error finding container 4bd01f360937c1674fb8a0e6b0ff3ed7a04b0896365523c6c873166efc08ac1b: Status 404 returned error can't find the container with id 4bd01f360937c1674fb8a0e6b0ff3ed7a04b0896365523c6c873166efc08ac1b
Jul 23 19:01:39.756583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:39.756551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f" event={"ID":"6f9cb974-5a29-4c83-a121-1752f84ee609","Type":"ContainerStarted","Data":"4bd01f360937c1674fb8a0e6b0ff3ed7a04b0896365523c6c873166efc08ac1b"}
Jul 23 19:01:41.281177 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.281143    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jqzqj"]
Jul 23 19:01:41.303142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.303109    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jqzqj"]
Jul 23 19:01:41.303291 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.303250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.305480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.305451    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 19:01:41.305617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.305532    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 19:01:41.305617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.305546    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 19:01:41.305771 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.305755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cdq4c\""
Jul 23 19:01:41.306269 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.306240    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 19:01:41.452061 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.452031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.452178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.452081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77b04f81-40f3-4a86-a02f-1fc31f770d1e-crio-socket\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.452178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.452106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.452247 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.452188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrbjk\" (UniqueName: \"kubernetes.io/projected/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-api-access-zrbjk\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.452247 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.452229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b04f81-40f3-4a86-a02f-1fc31f770d1e-data-volume\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.522725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.522700    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g8mmq"]
Jul 23 19:01:41.538261 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.538204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g8mmq"]
Jul 23 19:01:41.538355 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.538291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.541891 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.541871    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 23 19:01:41.541984 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.541968    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 23 19:01:41.542042 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.542030    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 23 19:01:41.542139 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.542123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qvxhh\""
Jul 23 19:01:41.542380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.542362    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 23 19:01:41.552898 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.552878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77b04f81-40f3-4a86-a02f-1fc31f770d1e-crio-socket\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553008 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.552909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553008 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:41.552997    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 19:01:41.553107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.553057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrbjk\" (UniqueName: \"kubernetes.io/projected/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-api-access-zrbjk\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.553099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b04f81-40f3-4a86-a02f-1fc31f770d1e-data-volume\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.553060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77b04f81-40f3-4a86-a02f-1fc31f770d1e-crio-socket\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553197 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:41.553132    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:42.053102824 +0000 UTC m=+44.955688892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : configmap "kube-rbac-proxy" not found
Jul 23 19:01:41.553197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.553187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.553318 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:41.553277    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:41.553318 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:41.553315    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:42.053305323 +0000 UTC m=+44.955891382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:41.553382 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.553370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b04f81-40f3-4a86-a02f-1fc31f770d1e-data-volume\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.565540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.565513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrbjk\" (UniqueName: \"kubernetes.io/projected/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-api-access-zrbjk\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:41.653484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.653462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-cabundle\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.653564 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.653538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-key\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.653599 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.653568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxjt\" (UniqueName: \"kubernetes.io/projected/7a6c0999-19fe-4a19-8d91-9930c4318888-kube-api-access-tgxjt\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.754515 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.754496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-key\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.754578 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.754537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxjt\" (UniqueName: \"kubernetes.io/projected/7a6c0999-19fe-4a19-8d91-9930c4318888-kube-api-access-tgxjt\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.754578 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.754573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-cabundle\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.755287 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.755265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-cabundle\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.756900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.756882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7a6c0999-19fe-4a19-8d91-9930c4318888-signing-key\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.763287 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.763259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f" event={"ID":"6f9cb974-5a29-4c83-a121-1752f84ee609","Type":"ContainerStarted","Data":"c5a70eae63952fdc4adad2f264aa733b6616279a6e4c1c892efd15bb4e0b5922"}
Jul 23 19:01:41.763692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.763674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxjt\" (UniqueName: \"kubernetes.io/projected/7a6c0999-19fe-4a19-8d91-9930c4318888-kube-api-access-tgxjt\") pod \"service-ca-58598695f7-g8mmq\" (UID: \"7a6c0999-19fe-4a19-8d91-9930c4318888\") " pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.849347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.849323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-g8mmq"
Jul 23 19:01:41.996110 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:41.996072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g8mmq"]
Jul 23 19:01:41.998513 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:01:41.998487    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6c0999_19fe_4a19_8d91_9930c4318888.slice/crio-73575e2ed177fc56ee6a4a8c152c07a1d25f0068f1fc518920a5f41a393cae91 WatchSource:0}: Error finding container 73575e2ed177fc56ee6a4a8c152c07a1d25f0068f1fc518920a5f41a393cae91: Status 404 returned error can't find the container with id 73575e2ed177fc56ee6a4a8c152c07a1d25f0068f1fc518920a5f41a393cae91
Jul 23 19:01:42.057511 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:42.057485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:42.057614 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:42.057535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:42.057658 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:42.057627    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:42.057658 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:42.057637    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 19:01:42.057721 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:42.057679    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:43.057664472 +0000 UTC m=+45.960250527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:42.057721 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:42.057693    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:43.057687165 +0000 UTC m=+45.960273222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : configmap "kube-rbac-proxy" not found
Jul 23 19:01:42.768029 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:42.767987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f" event={"ID":"6f9cb974-5a29-4c83-a121-1752f84ee609","Type":"ContainerStarted","Data":"8b9fe0426fbd77085ddb573f266d81500e72047273a7c5757b0661449119dc48"}
Jul 23 19:01:42.769383 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:42.769359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-g8mmq" event={"ID":"7a6c0999-19fe-4a19-8d91-9930c4318888","Type":"ContainerStarted","Data":"73575e2ed177fc56ee6a4a8c152c07a1d25f0068f1fc518920a5f41a393cae91"}
Jul 23 19:01:42.791166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:42.791127    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-c9d8f" podStartSLOduration=1.70656819 podStartE2EDuration="3.791112257s" totalImagesPullingTime="2.084544067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:42.789567076 +0000 UTC m=+45.692153234" watchObservedRunningTime="2026-07-23 19:01:42.791112257 +0000 UTC m=+45.693698334"
Jul 23 19:01:43.064267 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:43.064184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:43.064267 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:43.064253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:43.064449 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:43.064314    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 19:01:43.064449 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:43.064336    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:43.064449 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:43.064376    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:45.064362647 +0000 UTC m=+47.966948703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : configmap "kube-rbac-proxy" not found
Jul 23 19:01:43.064449 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:43.064389    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:45.064383215 +0000 UTC m=+47.966969271 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:43.833187 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:43.833157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcdmw_8bda1177-dc5b-41fd-849d-f590926553b7/dns-node-resolver/0.log"
Jul 23 19:01:44.778188 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:44.778153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-g8mmq" event={"ID":"7a6c0999-19fe-4a19-8d91-9930c4318888","Type":"ContainerStarted","Data":"ccf0321143f13d4007833eede2176852bd2c0ca2f7c8dae4a436e9c7ab96036b"}
Jul 23 19:01:44.837288 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:44.837264    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gm8mt_9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7/node-ca/0.log"
Jul 23 19:01:45.080239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:45.080167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:45.080239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:45.080217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:45.080447 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:45.080319    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 19:01:45.080447 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:45.080332    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:45.080447 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:45.080374    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:49.080358931 +0000 UTC m=+51.982944989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : configmap "kube-rbac-proxy" not found
Jul 23 19:01:45.080447 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:45.080387    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:49.080381191 +0000 UTC m=+51.982967246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:45.432433 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:45.432349    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dxxx6_44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c/serve-healthcheck-canary/0.log"
Jul 23 19:01:45.636748 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:45.636704    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-c9d8f_6f9cb974-5a29-4c83-a121-1752f84ee609/migrator/0.log"
Jul 23 19:01:45.830619 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:45.830596    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-c9d8f_6f9cb974-5a29-4c83-a121-1752f84ee609/graceful-termination/0.log"
Jul 23 19:01:46.794861 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:46.794828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:01:46.795330 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:46.794963    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 19:01:46.795330 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:46.794977    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-df69497d5-tv5cc: secret "image-registry-tls" not found
Jul 23 19:01:46.795330 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:46.795035    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls podName:9d775deb-2b5a-45d7-b748-71d7d3df2c4b nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.795017103 +0000 UTC m=+65.697603178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls") pod "image-registry-df69497d5-tv5cc" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b") : secret "image-registry-tls" not found
Jul 23 19:01:46.896040 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:46.896010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:01:46.896188 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:46.896167    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 19:01:46.896249 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:46.896238    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls podName:ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.896219635 +0000 UTC m=+65.798805695 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls") pod "dns-default-xg9q2" (UID: "ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740") : secret "dns-default-metrics-tls" not found
Jul 23 19:01:49.111803 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:49.111752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:49.112295 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:49.111910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:49.112295 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:49.112049    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:49.112295 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:01:49.112127    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls podName:77b04f81-40f3-4a86-a02f-1fc31f770d1e nodeName:}" failed. No retries permitted until 2026-07-23 19:01:57.11210779 +0000 UTC m=+60.014693850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqzqj" (UID: "77b04f81-40f3-4a86-a02f-1fc31f770d1e") : secret "insights-runtime-extractor-tls" not found
Jul 23 19:01:49.112295 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:49.112257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77b04f81-40f3-4a86-a02f-1fc31f770d1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:56.716491 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:56.716463    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmvgp"
Jul 23 19:01:56.747898 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:56.747849    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-g8mmq" podStartSLOduration=13.527437352 podStartE2EDuration="15.74783022s" totalImagesPullingTime="2.220392868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:44.795335192 +0000 UTC m=+47.697921270" watchObservedRunningTime="2026-07-23 19:01:56.74783022 +0000 UTC m=+59.650416290"
Jul 23 19:01:57.175177 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:57.175139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:57.177423 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:57.177400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77b04f81-40f3-4a86-a02f-1fc31f770d1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqzqj\" (UID: \"77b04f81-40f3-4a86-a02f-1fc31f770d1e\") " pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:57.214351 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:57.214322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jqzqj"
Jul 23 19:01:57.333571 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:57.333545    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jqzqj"]
Jul 23 19:01:57.336655 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:01:57.336628    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b04f81_40f3_4a86_a02f_1fc31f770d1e.slice/crio-9eae05ae60007d3b95064134398405aa1ac9b4ab30764ed7dabb85c5047deb1a WatchSource:0}: Error finding container 9eae05ae60007d3b95064134398405aa1ac9b4ab30764ed7dabb85c5047deb1a: Status 404 returned error can't find the container with id 9eae05ae60007d3b95064134398405aa1ac9b4ab30764ed7dabb85c5047deb1a
Jul 23 19:01:57.814245 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:01:57.814196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqzqj" event={"ID":"77b04f81-40f3-4a86-a02f-1fc31f770d1e","Type":"ContainerStarted","Data":"9eae05ae60007d3b95064134398405aa1ac9b4ab30764ed7dabb85c5047deb1a"}
Jul 23 19:02:02.219153 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.219119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:02:02.221402 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.221384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhn9b\" (UniqueName: \"kubernetes.io/projected/dcf73f6a-37e7-4997-a658-354c29cbabd0-kube-api-access-qhn9b\") pod \"network-check-target-4sbt5\" (UID: \"dcf73f6a-37e7-4997-a658-354c29cbabd0\") " pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:02:02.319549 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.319504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:02:02.319691 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.319569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:02:02.322266 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.322245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 19:02:02.322936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.322921    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 19:02:02.332767 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.332749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05c0c74-6f50-4551-a0c2-6c8da9b2eace-metrics-certs\") pod \"network-metrics-daemon-9s5gk\" (UID: \"d05c0c74-6f50-4551-a0c2-6c8da9b2eace\") " pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:02:02.332873 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.332855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e0324-af23-4a58-9836-35b49754f300-original-pull-secret\") pod \"global-pull-secret-syncer-fwvxh\" (UID: \"4a3e0324-af23-4a58-9836-35b49754f300\") " pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:02:02.427376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.427304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fwvxh"
Jul 23 19:02:02.438014 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.437988    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5ztj\""
Jul 23 19:02:02.442243 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.442225    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8qs8\""
Jul 23 19:02:02.446086 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.446069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5gk"
Jul 23 19:02:02.450963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.450944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:02:02.566346 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.566184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fwvxh"]
Jul 23 19:02:02.571441 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:02.571407    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3e0324_af23_4a58_9836_35b49754f300.slice/crio-c7a53f2b54a2f0f3e4ad04aaf25a42f7ba423084418a76f235f99367d69e08c6 WatchSource:0}: Error finding container c7a53f2b54a2f0f3e4ad04aaf25a42f7ba423084418a76f235f99367d69e08c6: Status 404 returned error can't find the container with id c7a53f2b54a2f0f3e4ad04aaf25a42f7ba423084418a76f235f99367d69e08c6
Jul 23 19:02:02.583640 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.583526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9s5gk"]
Jul 23 19:02:02.586305 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:02.586272    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05c0c74_6f50_4551_a0c2_6c8da9b2eace.slice/crio-9e154d33d350023d69b475be5adaac12f971c0df0e185c5568ef27c708f3b6be WatchSource:0}: Error finding container 9e154d33d350023d69b475be5adaac12f971c0df0e185c5568ef27c708f3b6be: Status 404 returned error can't find the container with id 9e154d33d350023d69b475be5adaac12f971c0df0e185c5568ef27c708f3b6be
Jul 23 19:02:02.597676 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.597654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sbt5"]
Jul 23 19:02:02.600463 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:02.600442    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf73f6a_37e7_4997_a658_354c29cbabd0.slice/crio-44b3af79d0a4af522c178160396fd2db673e5a8ace511c5dc18a8be5902f2843 WatchSource:0}: Error finding container 44b3af79d0a4af522c178160396fd2db673e5a8ace511c5dc18a8be5902f2843: Status 404 returned error can't find the container with id 44b3af79d0a4af522c178160396fd2db673e5a8ace511c5dc18a8be5902f2843
Jul 23 19:02:02.822204 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.822168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:02.824931 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.824894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"image-registry-df69497d5-tv5cc\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") " pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:02.830296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.829845    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sbt5" event={"ID":"dcf73f6a-37e7-4997-a658-354c29cbabd0","Type":"ContainerStarted","Data":"af1ab2e29f62a7ffa5b403a19000014d1eb8d94a4ecad4fdb9c660fb04077aa3"}
Jul 23 19:02:02.830296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.829905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sbt5" event={"ID":"dcf73f6a-37e7-4997-a658-354c29cbabd0","Type":"ContainerStarted","Data":"44b3af79d0a4af522c178160396fd2db673e5a8ace511c5dc18a8be5902f2843"}
Jul 23 19:02:02.830296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.829932    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:02:02.831722 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.831699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5gk" event={"ID":"d05c0c74-6f50-4551-a0c2-6c8da9b2eace","Type":"ContainerStarted","Data":"9e154d33d350023d69b475be5adaac12f971c0df0e185c5568ef27c708f3b6be"}
Jul 23 19:02:02.833050 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.833027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fwvxh" event={"ID":"4a3e0324-af23-4a58-9836-35b49754f300","Type":"ContainerStarted","Data":"c7a53f2b54a2f0f3e4ad04aaf25a42f7ba423084418a76f235f99367d69e08c6"}
Jul 23 19:02:02.834664 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.834643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqzqj" event={"ID":"77b04f81-40f3-4a86-a02f-1fc31f770d1e","Type":"ContainerStarted","Data":"14e89d33b67fc00d0c6a8b4894ffc5797599cfb1eb1c545ffe1edf8692330dd5"}
Jul 23 19:02:02.851365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.851325    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4sbt5" podStartSLOduration=65.851310743 podStartE2EDuration="1m5.851310743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:02.85061239 +0000 UTC m=+65.753198469" watchObservedRunningTime="2026-07-23 19:02:02.851310743 +0000 UTC m=+65.753896845"
Jul 23 19:02:02.923153 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.923117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:02:02.925716 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:02.925689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740-metrics-tls\") pod \"dns-default-xg9q2\" (UID: \"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740\") " pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:02:03.013390 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.013353    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"]
Jul 23 19:02:03.016289 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.016271    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"]
Jul 23 19:02:03.016411 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.016394    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.019237 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.019217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 23 19:02:03.019701 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.019685    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t4s75\""
Jul 23 19:02:03.019848 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.019830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 23 19:02:03.019913 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.019887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.019960 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.019886    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"]
Jul 23 19:02:03.023072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.022783    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.027316 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.027294    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 19:02:03.028378 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.028359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 19:02:03.028505 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.028486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 19:02:03.028581 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.028564    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 23 19:02:03.028700 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.028686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 19:02:03.028771 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.028755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6zd77\""
Jul 23 19:02:03.035286 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.034794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"]
Jul 23 19:02:03.035960 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.035938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"]
Jul 23 19:02:03.042166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.042147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"]
Jul 23 19:02:03.082957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.082867    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:02:03.083969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.083743    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nv7r4\""
Jul 23 19:02:03.092408 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.092386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:03.118685 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.118447    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fgf46\""
Jul 23 19:02:03.120770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.119739    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-2wh8f"]
Jul 23 19:02:03.123121 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.123099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ebdf231-f22c-4a3a-becb-588570315da2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6a67d243-4946-4f46-9877-41c0efdb94d1-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67d243-4946-4f46-9877-41c0efdb94d1-tmp\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chn4\" (UniqueName: \"kubernetes.io/projected/3ebdf231-f22c-4a3a-becb-588570315da2-kube-api-access-8chn4\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lrd\" (UniqueName: \"kubernetes.io/projected/6a67d243-4946-4f46-9877-41c0efdb94d1-kube-api-access-67lrd\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf4fa8f2-012d-4e36-9925-2f09feba437a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.124966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.124420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bf4fa8f2-012d-4e36-9925-2f09feba437a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.125603 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.125447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:02:03.127252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.126687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8jd79\""
Jul 23 19:02:03.127252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.126906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 19:02:03.127252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.127033    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 19:02:03.135770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.135707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-2wh8f"]
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.228102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chn4\" (UniqueName: \"kubernetes.io/projected/3ebdf231-f22c-4a3a-becb-588570315da2-kube-api-access-8chn4\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.228172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lrd\" (UniqueName: \"kubernetes.io/projected/6a67d243-4946-4f46-9877-41c0efdb94d1-kube-api-access-67lrd\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.228206    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"]
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.228941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf4fa8f2-012d-4e36-9925-2f09feba437a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.228206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf4fa8f2-012d-4e36-9925-2f09feba437a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.229477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bf4fa8f2-012d-4e36-9925-2f09feba437a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.229530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ebdf231-f22c-4a3a-becb-588570315da2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.229562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6a67d243-4946-4f46-9877-41c0efdb94d1-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.229591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67d243-4946-4f46-9877-41c0efdb94d1-tmp\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.230342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.229619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw5p7\" (UniqueName: \"kubernetes.io/projected/3a96e80c-5555-42f4-9d10-dfb55851a3bb-kube-api-access-xw5p7\") pod \"downloads-5bc67dcd46-2wh8f\" (UID: \"3a96e80c-5555-42f4-9d10-dfb55851a3bb\") " pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:03.231505 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.230531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a67d243-4946-4f46-9877-41c0efdb94d1-tmp\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.232826 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.232798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.236283 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.236151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6a67d243-4946-4f46-9877-41c0efdb94d1-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.236283 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.236226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bf4fa8f2-012d-4e36-9925-2f09feba437a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kwtnv\" (UID: \"bf4fa8f2-012d-4e36-9925-2f09feba437a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.236651 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.236581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ebdf231-f22c-4a3a-becb-588570315da2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.246382 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.246312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"]
Jul 23 19:02:03.252201 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.252062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chn4\" (UniqueName: \"kubernetes.io/projected/3ebdf231-f22c-4a3a-becb-588570315da2-kube-api-access-8chn4\") pod \"managed-serviceaccount-addon-agent-68747f7ffd-fk492\" (UID: \"3ebdf231-f22c-4a3a-becb-588570315da2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.252751 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.252725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lrd\" (UniqueName: \"kubernetes.io/projected/6a67d243-4946-4f46-9877-41c0efdb94d1-kube-api-access-67lrd\") pod \"klusterlet-addon-workmgr-549bccc944-52zdk\" (UID: \"6a67d243-4946-4f46-9877-41c0efdb94d1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.286502 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.286431    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:02:03.296145 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:03.292891    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d775deb_2b5a_45d7_b748_71d7d3df2c4b.slice/crio-6a7ff643471684b52629f4907e8feae6c188579184d3c54e8d282341ea070d9f WatchSource:0}: Error finding container 6a7ff643471684b52629f4907e8feae6c188579184d3c54e8d282341ea070d9f: Status 404 returned error can't find the container with id 6a7ff643471684b52629f4907e8feae6c188579184d3c54e8d282341ea070d9f
Jul 23 19:02:03.327188 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.327133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-bound-sa-token\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-certificates\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29e33cca-e56c-47d5-8796-81f26f1ab1a2-ca-trust-extracted\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjnvg\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-kube-api-access-zjnvg\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-installation-pull-secrets\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-image-registry-private-configuration\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-tls\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw5p7\" (UniqueName: \"kubernetes.io/projected/3a96e80c-5555-42f4-9d10-dfb55851a3bb-kube-api-access-xw5p7\") pod \"downloads-5bc67dcd46-2wh8f\" (UID: \"3a96e80c-5555-42f4-9d10-dfb55851a3bb\") " pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:03.333135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.332713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-trusted-ca\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.334670 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.334157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xg9q2"]
Jul 23 19:02:03.335044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.335021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:03.339747 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:03.339717    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6a0efc_a9f2_481d_89c9_cfe2dd8cd740.slice/crio-415937a1dd8cabc695480a08c224ca4dcb00823e43c0e727e55785611cbef75b WatchSource:0}: Error finding container 415937a1dd8cabc695480a08c224ca4dcb00823e43c0e727e55785611cbef75b: Status 404 returned error can't find the container with id 415937a1dd8cabc695480a08c224ca4dcb00823e43c0e727e55785611cbef75b
Jul 23 19:02:03.349129 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.349105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw5p7\" (UniqueName: \"kubernetes.io/projected/3a96e80c-5555-42f4-9d10-dfb55851a3bb-kube-api-access-xw5p7\") pod \"downloads-5bc67dcd46-2wh8f\" (UID: \"3a96e80c-5555-42f4-9d10-dfb55851a3bb\") " pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:03.353637 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.353589    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"
Jul 23 19:02:03.438284 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29e33cca-e56c-47d5-8796-81f26f1ab1a2-ca-trust-extracted\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438284 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjnvg\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-kube-api-access-zjnvg\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438284 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-installation-pull-secrets\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438284 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-image-registry-private-configuration\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-tls\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-trusted-ca\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-bound-sa-token\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.438615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.438409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-certificates\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.441182 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.439319    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-certificates\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.441182 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.439606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29e33cca-e56c-47d5-8796-81f26f1ab1a2-ca-trust-extracted\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.443261 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.443211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29e33cca-e56c-47d5-8796-81f26f1ab1a2-trusted-ca\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.449359 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.448506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-image-registry-private-configuration\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.452450 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.451977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-registry-tls\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.452450 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.452412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29e33cca-e56c-47d5-8796-81f26f1ab1a2-installation-pull-secrets\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.466631 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.466606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:03.472748 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.472696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjnvg\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-kube-api-access-zjnvg\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.479987 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.479931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29e33cca-e56c-47d5-8796-81f26f1ab1a2-bound-sa-token\") pod \"image-registry-7dfd8c47f4-6mgfg\" (UID: \"29e33cca-e56c-47d5-8796-81f26f1ab1a2\") " pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.565020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.564715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:03.570551 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.570438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"]
Jul 23 19:02:03.595682 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.595146    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492"]
Jul 23 19:02:03.678736 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.678638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-2wh8f"]
Jul 23 19:02:03.763765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.763728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"]
Jul 23 19:02:03.792668 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.792636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv"]
Jul 23 19:02:03.842791 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.842084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" event={"ID":"9d775deb-2b5a-45d7-b748-71d7d3df2c4b","Type":"ContainerStarted","Data":"50ef7b96d8f1f7eeef28130774f25e0d9bc2db6d1205b6a1ca6a9a5bc56a68cd"}
Jul 23 19:02:03.842791 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.842147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" event={"ID":"9d775deb-2b5a-45d7-b748-71d7d3df2c4b","Type":"ContainerStarted","Data":"6a7ff643471684b52629f4907e8feae6c188579184d3c54e8d282341ea070d9f"}
Jul 23 19:02:03.842791 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.842380    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:03.844887 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.844854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xg9q2" event={"ID":"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740","Type":"ContainerStarted","Data":"415937a1dd8cabc695480a08c224ca4dcb00823e43c0e727e55785611cbef75b"}
Jul 23 19:02:03.871244 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:03.871159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" podStartSLOduration=35.871131677 podStartE2EDuration="35.871131677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:03.870018716 +0000 UTC m=+66.772604800" watchObservedRunningTime="2026-07-23 19:02:03.871131677 +0000 UTC m=+66.773717755"
Jul 23 19:02:04.032216 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:04.032179    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a96e80c_5555_42f4_9d10_dfb55851a3bb.slice/crio-5f07d9f8dac320c9d6ca6370256d024ec3da42cbb1fb5b77049bdbdcef6dd02b WatchSource:0}: Error finding container 5f07d9f8dac320c9d6ca6370256d024ec3da42cbb1fb5b77049bdbdcef6dd02b: Status 404 returned error can't find the container with id 5f07d9f8dac320c9d6ca6370256d024ec3da42cbb1fb5b77049bdbdcef6dd02b
Jul 23 19:02:04.033934 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:04.033855    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ebdf231_f22c_4a3a_becb_588570315da2.slice/crio-0efbf565d14710c3aebbecd5941dbf0a9531c7dd1ffbe4c471927906cae03b05 WatchSource:0}: Error finding container 0efbf565d14710c3aebbecd5941dbf0a9531c7dd1ffbe4c471927906cae03b05: Status 404 returned error can't find the container with id 0efbf565d14710c3aebbecd5941dbf0a9531c7dd1ffbe4c471927906cae03b05
Jul 23 19:02:04.034206 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:04.034186    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4fa8f2_012d_4e36_9925_2f09feba437a.slice/crio-0962f6298ce7cd6009f9ee0309653778bbea8365354ea33f7c1fc339a1847558 WatchSource:0}: Error finding container 0962f6298ce7cd6009f9ee0309653778bbea8365354ea33f7c1fc339a1847558: Status 404 returned error can't find the container with id 0962f6298ce7cd6009f9ee0309653778bbea8365354ea33f7c1fc339a1847558
Jul 23 19:02:04.856110 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.855239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5gk" event={"ID":"d05c0c74-6f50-4551-a0c2-6c8da9b2eace","Type":"ContainerStarted","Data":"4b2311a59c93125a3b664150ec90ac3fd4403c49e3501495652e1bc867d6fe95"}
Jul 23 19:02:04.856110 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.855305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5gk" event={"ID":"d05c0c74-6f50-4551-a0c2-6c8da9b2eace","Type":"ContainerStarted","Data":"b5583146de578423bfd026dc59e8596dca438c7ed2e235d6cf2bb56367c27783"}
Jul 23 19:02:04.859562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.859238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg" event={"ID":"29e33cca-e56c-47d5-8796-81f26f1ab1a2","Type":"ContainerStarted","Data":"a4e1ec7f66bd2e0610fb4c2eed735893d3161260b70822bbc0162b1e97a58167"}
Jul 23 19:02:04.859562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.859287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg" event={"ID":"29e33cca-e56c-47d5-8796-81f26f1ab1a2","Type":"ContainerStarted","Data":"6428f6e4f28aadb729068e29b9f7465eb7c0da28ff8aeffd948931beb360810e"}
Jul 23 19:02:04.860217 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.860061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:04.861935 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.861890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-2wh8f" event={"ID":"3a96e80c-5555-42f4-9d10-dfb55851a3bb","Type":"ContainerStarted","Data":"5f07d9f8dac320c9d6ca6370256d024ec3da42cbb1fb5b77049bdbdcef6dd02b"}
Jul 23 19:02:04.863451 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.863388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492" event={"ID":"3ebdf231-f22c-4a3a-becb-588570315da2","Type":"ContainerStarted","Data":"0efbf565d14710c3aebbecd5941dbf0a9531c7dd1ffbe4c471927906cae03b05"}
Jul 23 19:02:04.867746 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.867159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk" event={"ID":"6a67d243-4946-4f46-9877-41c0efdb94d1","Type":"ContainerStarted","Data":"11870c9b7741c7f21e03f76b14c5899af9e88e03a8242c25c6d6d8a9b939f473"}
Jul 23 19:02:04.868877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.868853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv" event={"ID":"bf4fa8f2-012d-4e36-9925-2f09feba437a","Type":"ContainerStarted","Data":"0962f6298ce7cd6009f9ee0309653778bbea8365354ea33f7c1fc339a1847558"}
Jul 23 19:02:04.881498 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.880848    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9s5gk" podStartSLOduration=66.383668432 podStartE2EDuration="1m7.880832527s" totalImagesPullingTime="1.497164095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:04.880500647 +0000 UTC m=+67.783086724" watchObservedRunningTime="2026-07-23 19:02:04.880832527 +0000 UTC m=+67.783418604"
Jul 23 19:02:04.905056 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:04.905001    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg" podStartSLOduration=1.904987516 podStartE2EDuration="1.904987516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:04.904148837 +0000 UTC m=+67.806734942" watchObservedRunningTime="2026-07-23 19:02:04.904987516 +0000 UTC m=+67.807573593"
Jul 23 19:02:13.914216 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.914178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqzqj" event={"ID":"77b04f81-40f3-4a86-a02f-1fc31f770d1e","Type":"ContainerStarted","Data":"d2cd0d2cde6b30df31c87185d7ae154442f68ccb2b9036312b8b1ed0a4079174"}
Jul 23 19:02:13.916432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.916342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fwvxh" event={"ID":"4a3e0324-af23-4a58-9836-35b49754f300","Type":"ContainerStarted","Data":"ba931ff782be13f5afde1a36b43bc6a7101a1cdc0e57c61bbf151e9c26cfbbc0"}
Jul 23 19:02:13.918093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.918063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492" event={"ID":"3ebdf231-f22c-4a3a-becb-588570315da2","Type":"ContainerStarted","Data":"87316f16cd974daa546709ceb4f80d2a41f0db365cf04056cb360a241fbf29d6"}
Jul 23 19:02:13.920100 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.920071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk" event={"ID":"6a67d243-4946-4f46-9877-41c0efdb94d1","Type":"ContainerStarted","Data":"843bcb321c2cedd9d26df102d087b5e20a8225dfad7c54479061f47966383c55"}
Jul 23 19:02:13.920315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.920275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:13.921808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.921759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv" event={"ID":"bf4fa8f2-012d-4e36-9925-2f09feba437a","Type":"ContainerStarted","Data":"e133db647ed335bb30a61ca0f01cf2b019f8ab256676eeb4b351979b627aad74"}
Jul 23 19:02:13.922302 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.922284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk"
Jul 23 19:02:13.923874 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.923855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xg9q2" event={"ID":"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740","Type":"ContainerStarted","Data":"ba2bdb000c2bce02ba4de553711e77ac75302ce5cb72ceef362e8d43934e5c96"}
Jul 23 19:02:13.923974 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.923898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xg9q2" event={"ID":"ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740","Type":"ContainerStarted","Data":"3ce7f6cee0e4ce6846457d8e1eb6c0122a679e6794ca2a52d347f694adb96692"}
Jul 23 19:02:13.924070 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.924050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:02:13.939209 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.939126    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fwvxh" podStartSLOduration=66.07413223 podStartE2EDuration="1m16.939114327s" totalImagesPullingTime="10.864982097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.937840081 +0000 UTC m=+76.840426183" watchObservedRunningTime="2026-07-23 19:02:13.939114327 +0000 UTC m=+76.841700428"
Jul 23 19:02:13.965277 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.965233    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68747f7ffd-fk492" podStartSLOduration=2.559356758 podStartE2EDuration="11.965220036s" totalImagesPullingTime="9.405863278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.964973985 +0000 UTC m=+76.867560063" watchObservedRunningTime="2026-07-23 19:02:13.965220036 +0000 UTC m=+76.867806117"
Jul 23 19:02:13.987191 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:13.987150    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kwtnv" podStartSLOduration=2.589347521 podStartE2EDuration="11.987137961s" totalImagesPullingTime="9.39779044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.986191631 +0000 UTC m=+76.888777710" watchObservedRunningTime="2026-07-23 19:02:13.987137961 +0000 UTC m=+76.889724036"
Jul 23 19:02:14.007378 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:14.007336    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bccc944-52zdk" podStartSLOduration=2.589762823 podStartE2EDuration="12.007324158s" totalImagesPullingTime="9.417561335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:14.007253069 +0000 UTC m=+76.909839149" watchObservedRunningTime="2026-07-23 19:02:14.007324158 +0000 UTC m=+76.909910237"
Jul 23 19:02:14.032828 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:14.032769    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xg9q2" podStartSLOduration=33.938986026 podStartE2EDuration="44.032750908s" totalImagesPullingTime="10.093764882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:14.031901663 +0000 UTC m=+76.934487765" watchObservedRunningTime="2026-07-23 19:02:14.032750908 +0000 UTC m=+76.935336990"
Jul 23 19:02:23.570372 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:23.570239    2563 patch_prober.go:28] interesting pod/image-registry-7dfd8c47f4-6mgfg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 19:02:23.570372 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:23.570322    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg" podUID="29e33cca-e56c-47d5-8796-81f26f1ab1a2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 19:02:23.848375 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:23.848301    2563 patch_prober.go:28] interesting pod/image-registry-df69497d5-tv5cc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 19:02:23.848375 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:23.848365    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 19:02:23.929762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:23.929732    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xg9q2"
Jul 23 19:02:24.399206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.399173    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qh7zb"]
Jul 23 19:02:24.466751 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.466721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.470054 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470033    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 19:02:24.470179 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470031    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 19:02:24.470693 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470676    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 19:02:24.470762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470687    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 19:02:24.470762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470678    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 19:02:24.470972 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.470952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 19:02:24.471111 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.471002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b6jgk\""
Jul 23 19:02:24.621041 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-tls\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-root\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-metrics-client-ca\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7glj\" (UniqueName: \"kubernetes.io/projected/4d2540a2-73c6-4f3f-895b-62856e50d2b5-kube-api-access-n7glj\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-wtmp\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-sys\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.621466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.621255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-textfile\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-root\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-metrics-client-ca\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-root\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7glj\" (UniqueName: \"kubernetes.io/projected/4d2540a2-73c6-4f3f-895b-62856e50d2b5-kube-api-access-n7glj\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-wtmp\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-sys\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-textfile\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-wtmp\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.722919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-tls\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.723102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.723102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.722924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2540a2-73c6-4f3f-895b-62856e50d2b5-sys\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.723102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.723076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-textfile\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.723247 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.723201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-metrics-client-ca\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.723485 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.723463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.724945 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.724928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.725033 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.725024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4d2540a2-73c6-4f3f-895b-62856e50d2b5-node-exporter-tls\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.734762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.734740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7glj\" (UniqueName: \"kubernetes.io/projected/4d2540a2-73c6-4f3f-895b-62856e50d2b5-kube-api-access-n7glj\") pod \"node-exporter-qh7zb\" (UID: \"4d2540a2-73c6-4f3f-895b-62856e50d2b5\") " pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.776853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.776831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qh7zb"
Jul 23 19:02:24.957129 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:02:24.957100    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d2540a2_73c6_4f3f_895b_62856e50d2b5.slice/crio-5e6faceb310ed3b83d6fcc8a56fb918b2dac4d1b8720bcd1460567dee6fbf11f WatchSource:0}: Error finding container 5e6faceb310ed3b83d6fcc8a56fb918b2dac4d1b8720bcd1460567dee6fbf11f: Status 404 returned error can't find the container with id 5e6faceb310ed3b83d6fcc8a56fb918b2dac4d1b8720bcd1460567dee6fbf11f
Jul 23 19:02:24.970630 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:24.970571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh7zb" event={"ID":"4d2540a2-73c6-4f3f-895b-62856e50d2b5","Type":"ContainerStarted","Data":"5e6faceb310ed3b83d6fcc8a56fb918b2dac4d1b8720bcd1460567dee6fbf11f"}
Jul 23 19:02:25.977556 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:25.977518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-2wh8f" event={"ID":"3a96e80c-5555-42f4-9d10-dfb55851a3bb","Type":"ContainerStarted","Data":"8420adc45e07cba099e4143a334e00a02c905d1c030c708043a4b82e6082abbe"}
Jul 23 19:02:25.978033 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:25.977582    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:25.981436 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:25.981407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqzqj" event={"ID":"77b04f81-40f3-4a86-a02f-1fc31f770d1e","Type":"ContainerStarted","Data":"296b08bfce798ce171512e6bd7c50b6041f1151ca9fa45ea65aab5fc28e78217"}
Jul 23 19:02:25.989067 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:25.989046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-2wh8f"
Jul 23 19:02:26.001431 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:26.001241    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-2wh8f" podStartSLOduration=2.036736955 podStartE2EDuration="23.00122332s" totalImagesPullingTime="20.964486365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:25.999965276 +0000 UTC m=+88.902551385" watchObservedRunningTime="2026-07-23 19:02:26.00122332 +0000 UTC m=+88.903809399"
Jul 23 19:02:26.024011 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:26.023959    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jqzqj" podStartSLOduration=17.596378372 podStartE2EDuration="45.02394161s" totalImagesPullingTime="27.427563238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:01:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:26.022726473 +0000 UTC m=+88.925312569" watchObservedRunningTime="2026-07-23 19:02:26.02394161 +0000 UTC m=+88.926527689"
Jul 23 19:02:26.888765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:26.888732    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dfd8c47f4-6mgfg"
Jul 23 19:02:26.989012 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:26.988966    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d2540a2-73c6-4f3f-895b-62856e50d2b5" containerID="b4e7c1515b66afba1b786be8ec7805b656f83390c6dd25f43a575a7091157b30" exitCode=0
Jul 23 19:02:26.989464 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:26.989062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh7zb" event={"ID":"4d2540a2-73c6-4f3f-895b-62856e50d2b5","Type":"ContainerDied","Data":"b4e7c1515b66afba1b786be8ec7805b656f83390c6dd25f43a575a7091157b30"}
Jul 23 19:02:27.996089 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:27.996045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh7zb" event={"ID":"4d2540a2-73c6-4f3f-895b-62856e50d2b5","Type":"ContainerStarted","Data":"a73aa9372ad4aedd27e1e9725781cb85c12c04ca215dbfab7e178b54f8f2b3fa"}
Jul 23 19:02:27.996645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:27.996119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh7zb" event={"ID":"4d2540a2-73c6-4f3f-895b-62856e50d2b5","Type":"ContainerStarted","Data":"ef706ae0dffd3f2de92e2a660f8dc81515fbf69f5286a54b3e8341bc4121f12b"}
Jul 23 19:02:28.024442 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:28.024383    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qh7zb" podStartSLOduration=2.185970126 podStartE2EDuration="4.024364161s" totalImagesPullingTime="1.743367665s" totalInitContainerRuntime="95.02637ms" isStatefulPod=true podCreationTimestamp="2026-07-23 19:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:28.023927297 +0000 UTC m=+90.926513395" watchObservedRunningTime="2026-07-23 19:02:28.024364161 +0000 UTC m=+90.926950240"
Jul 23 19:02:28.876774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:28.876716    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerName="registry" containerID="cri-o://50ef7b96d8f1f7eeef28130774f25e0d9bc2db6d1205b6a1ca6a9a5bc56a68cd" gracePeriod=30
Jul 23 19:02:29.007360 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.007332    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerID="50ef7b96d8f1f7eeef28130774f25e0d9bc2db6d1205b6a1ca6a9a5bc56a68cd" exitCode=0
Jul 23 19:02:29.007725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.007378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" event={"ID":"9d775deb-2b5a-45d7-b748-71d7d3df2c4b","Type":"ContainerDied","Data":"50ef7b96d8f1f7eeef28130774f25e0d9bc2db6d1205b6a1ca6a9a5bc56a68cd"}
Jul 23 19:02:29.173341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.173313    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:29.261355 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261442    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bj7c\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261513    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261548    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.261780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261600    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token\") pod \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\" (UID: \"9d775deb-2b5a-45d7-b748-71d7d3df2c4b\") "
Jul 23 19:02:29.262222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261943    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:02:29.262222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.261952    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:02:29.264954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.264901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:02:29.264954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.264922    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:02:29.265129 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.265096    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:02:29.265342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.265315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:02:29.265342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.265318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c" (OuterVolumeSpecName: "kube-api-access-8bj7c") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "kube-api-access-8bj7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:02:29.272102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.272071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9d775deb-2b5a-45d7-b748-71d7d3df2c4b" (UID: "9d775deb-2b5a-45d7-b748-71d7d3df2c4b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:02:29.362821 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362785    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362833    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-installation-pull-secrets\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362852    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-ca-trust-extracted\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362866    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-trusted-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362876    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bj7c\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-kube-api-access-8bj7c\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362889    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-registry-certificates\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362905    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-image-registry-private-configuration\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:29.362955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:29.362920    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d775deb-2b5a-45d7-b748-71d7d3df2c4b-bound-sa-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:02:30.012742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:30.012705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-df69497d5-tv5cc" event={"ID":"9d775deb-2b5a-45d7-b748-71d7d3df2c4b","Type":"ContainerDied","Data":"6a7ff643471684b52629f4907e8feae6c188579184d3c54e8d282341ea070d9f"}
Jul 23 19:02:30.012742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:30.012728    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-df69497d5-tv5cc"
Jul 23 19:02:30.013257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:30.012777    2563 scope.go:117] "RemoveContainer" containerID="50ef7b96d8f1f7eeef28130774f25e0d9bc2db6d1205b6a1ca6a9a5bc56a68cd"
Jul 23 19:02:30.053640 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:30.053611    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:02:30.077412 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:30.077383    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-df69497d5-tv5cc"]
Jul 23 19:02:31.521565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:31.521524    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" path="/var/lib/kubelet/pods/9d775deb-2b5a-45d7-b748-71d7d3df2c4b/volumes"
Jul 23 19:02:33.847386 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:02:33.847355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4sbt5"
Jul 23 19:05:57.407316 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:05:57.407288    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 19:06:15.288396 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.288360    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"]
Jul 23 19:06:15.290742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.288674    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerName="registry"
Jul 23 19:06:15.290742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.288685    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerName="registry"
Jul 23 19:06:15.290742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.288734    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d775deb-2b5a-45d7-b748-71d7d3df2c4b" containerName="registry"
Jul 23 19:06:15.291657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.291640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.294282 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.294259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g4scf\""
Jul 23 19:06:15.295273 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.294941    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 19:06:15.298302 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.295540    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 19:06:15.314643 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.314611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"]
Jul 23 19:06:15.421744 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.421712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g762g\" (UniqueName: \"kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.421932 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.421754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.421932 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.421786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.522926 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.522898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g762g\" (UniqueName: \"kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.523086 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.522945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.523086 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.522989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.523347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.523326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.523411 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.523356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.533366 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.533339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g762g\" (UniqueName: \"kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.602864 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.602789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:15.735229 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.735194    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"]
Jul 23 19:06:15.739084 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:06:15.739055    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda771cc1c_7006_4b17_a031_ceb36bc2bf64.slice/crio-4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a WatchSource:0}: Error finding container 4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a: Status 404 returned error can't find the container with id 4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a
Jul 23 19:06:15.741006 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.740990    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:06:15.776310 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:15.776286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerStarted","Data":"4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a"}
Jul 23 19:06:22.805231 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:22.805193    2563 generic.go:356] "Generic (PLEG): container finished" podID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerID="c5e00f6246fdd7643e4a6e37c35125a21adbfa9398d010342152f09822fcd58c" exitCode=0
Jul 23 19:06:22.805603 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:22.805285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerDied","Data":"c5e00f6246fdd7643e4a6e37c35125a21adbfa9398d010342152f09822fcd58c"}
Jul 23 19:06:25.816547 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:25.816513    2563 generic.go:356] "Generic (PLEG): container finished" podID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerID="2dfbb6a77aa5521c4e4eea902a0ec4dac1155a2472c62258a513d34073c148a2" exitCode=0
Jul 23 19:06:25.816933 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:25.816574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerDied","Data":"2dfbb6a77aa5521c4e4eea902a0ec4dac1155a2472c62258a513d34073c148a2"}
Jul 23 19:06:33.847431 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:33.847389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerStarted","Data":"f0686f5a2816ab677b7371003ae85a29eabe68413eab0b4cb4b60f3a0683534a"}
Jul 23 19:06:33.873279 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:33.873220    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" podStartSLOduration=1.9005166839999998 podStartE2EDuration="18.873202854s" totalImagesPullingTime="16.97268617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:06:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:06:33.871577886 +0000 UTC m=+336.774163987" watchObservedRunningTime="2026-07-23 19:06:33.873202854 +0000 UTC m=+336.775788932"
Jul 23 19:06:34.852328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:34.852295    2563 generic.go:356] "Generic (PLEG): container finished" podID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerID="f0686f5a2816ab677b7371003ae85a29eabe68413eab0b4cb4b60f3a0683534a" exitCode=0
Jul 23 19:06:34.852709 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:34.852392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerDied","Data":"f0686f5a2816ab677b7371003ae85a29eabe68413eab0b4cb4b60f3a0683534a"}
Jul 23 19:06:35.973347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:35.973321    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:36.095369 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.095334    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle\") pod \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") "
Jul 23 19:06:36.095369 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.095371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util\") pod \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") "
Jul 23 19:06:36.095584 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.095422    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g762g\" (UniqueName: \"kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g\") pod \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\" (UID: \"a771cc1c-7006-4b17-a031-ceb36bc2bf64\") "
Jul 23 19:06:36.096016 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.095991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle" (OuterVolumeSpecName: "bundle") pod "a771cc1c-7006-4b17-a031-ceb36bc2bf64" (UID: "a771cc1c-7006-4b17-a031-ceb36bc2bf64"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:06:36.097590 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.097567    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g" (OuterVolumeSpecName: "kube-api-access-g762g") pod "a771cc1c-7006-4b17-a031-ceb36bc2bf64" (UID: "a771cc1c-7006-4b17-a031-ceb36bc2bf64"). InnerVolumeSpecName "kube-api-access-g762g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:06:36.099857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.099832    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util" (OuterVolumeSpecName: "util") pod "a771cc1c-7006-4b17-a031-ceb36bc2bf64" (UID: "a771cc1c-7006-4b17-a031-ceb36bc2bf64"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:06:36.196909 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.196790    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:36.196909 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.196854    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a771cc1c-7006-4b17-a031-ceb36bc2bf64-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:36.196909 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.196865    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g762g\" (UniqueName: \"kubernetes.io/projected/a771cc1c-7006-4b17-a031-ceb36bc2bf64-kube-api-access-g762g\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:36.862250 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.862214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7" event={"ID":"a771cc1c-7006-4b17-a031-ceb36bc2bf64","Type":"ContainerDied","Data":"4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a"}
Jul 23 19:06:36.862415 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.862255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408ldh7"
Jul 23 19:06:36.862415 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:36.862264    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f43cff6c8cbd28755912215b517dc8a182f0ccd76ffaffcd720b758c5d6ea6a"
Jul 23 19:06:49.410313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410276    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w7gp9"]
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410538    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="pull"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410549    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="pull"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410564    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="util"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410569    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="util"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410581    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="extract"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="extract"
Jul 23 19:06:49.410754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.410661    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a771cc1c-7006-4b17-a031-ceb36bc2bf64" containerName="extract"
Jul 23 19:06:49.416183 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.416166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.420952 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.420928    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 19:06:49.421631 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.421611    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jmsqv\""
Jul 23 19:06:49.421714 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.421636    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 19:06:49.442744 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.442717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w7gp9"]
Jul 23 19:06:49.489969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.489929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvqwh\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-kube-api-access-hvqwh\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.489969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.489973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.591063 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.591030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvqwh\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-kube-api-access-hvqwh\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.591063 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.591073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.605806 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.605776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.605981 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.605961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvqwh\" (UniqueName: \"kubernetes.io/projected/4f654cdf-7688-473c-b066-3893b583fcda-kube-api-access-hvqwh\") pod \"cert-manager-webhook-6486ffd997-w7gp9\" (UID: \"4f654cdf-7688-473c-b066-3893b583fcda\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.724477 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.724390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:49.884920 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.884891    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w7gp9"]
Jul 23 19:06:49.887176 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:06:49.887150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f654cdf_7688_473c_b066_3893b583fcda.slice/crio-52af677a2aaeeb437f5ed8a2756d92a9f9412b87f0061ce57dee4b87547c4567 WatchSource:0}: Error finding container 52af677a2aaeeb437f5ed8a2756d92a9f9412b87f0061ce57dee4b87547c4567: Status 404 returned error can't find the container with id 52af677a2aaeeb437f5ed8a2756d92a9f9412b87f0061ce57dee4b87547c4567
Jul 23 19:06:49.905793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:49.905767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9" event={"ID":"4f654cdf-7688-473c-b066-3893b583fcda","Type":"ContainerStarted","Data":"52af677a2aaeeb437f5ed8a2756d92a9f9412b87f0061ce57dee4b87547c4567"}
Jul 23 19:06:50.088686 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.088639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"]
Jul 23 19:06:50.093108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.093084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.095437 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.095409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.095573 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.095456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lwg\" (UniqueName: \"kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.095573 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.095558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.095852 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.095793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 19:06:50.096667 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.096648    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 19:06:50.096754 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.096679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g4scf\""
Jul 23 19:06:50.102010 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.101982    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"]
Jul 23 19:06:50.196222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.196194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.196412 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.196234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.196412 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.196271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lwg\" (UniqueName: \"kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.196651 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.196629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.196711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.196651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.206607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.206584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lwg\" (UniqueName: \"kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.402889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.402775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:50.547039 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.547016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"]
Jul 23 19:06:50.549307 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:06:50.549279    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd503566d_7b8f_410f_93ff_2a68a171c406.slice/crio-4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa WatchSource:0}: Error finding container 4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa: Status 404 returned error can't find the container with id 4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa
Jul 23 19:06:50.913144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.913108    2563 generic.go:356] "Generic (PLEG): container finished" podID="d503566d-7b8f-410f-93ff-2a68a171c406" containerID="2b289e16ee3f4d78aee809c77770b61f1e7a1c49630b6b56e12da50ac21ebee4" exitCode=0
Jul 23 19:06:50.913313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.913200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm" event={"ID":"d503566d-7b8f-410f-93ff-2a68a171c406","Type":"ContainerDied","Data":"2b289e16ee3f4d78aee809c77770b61f1e7a1c49630b6b56e12da50ac21ebee4"}
Jul 23 19:06:50.913313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:50.913264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm" event={"ID":"d503566d-7b8f-410f-93ff-2a68a171c406","Type":"ContainerStarted","Data":"4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa"}
Jul 23 19:06:53.927325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:53.927289    2563 generic.go:356] "Generic (PLEG): container finished" podID="d503566d-7b8f-410f-93ff-2a68a171c406" containerID="d2950e7172b4a1e31cb11d0d94b2bc6f36335df2ec2fb98a42b4e19511b3a10e" exitCode=0
Jul 23 19:06:53.927750 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:53.927350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm" event={"ID":"d503566d-7b8f-410f-93ff-2a68a171c406","Type":"ContainerDied","Data":"d2950e7172b4a1e31cb11d0d94b2bc6f36335df2ec2fb98a42b4e19511b3a10e"}
Jul 23 19:06:53.929054 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:53.929029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9" event={"ID":"4f654cdf-7688-473c-b066-3893b583fcda","Type":"ContainerStarted","Data":"a9d1fc3f8e36185454270bd53e6bf059c2a0ea4ab79568ab3d1112df7dd7ff9f"}
Jul 23 19:06:53.929166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:53.929135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:06:53.969170 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:53.969113    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9" podStartSLOduration=1.575181477 podStartE2EDuration="4.969093932s" totalImagesPullingTime="3.393912455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:06:53.968181898 +0000 UTC m=+356.870767980" watchObservedRunningTime="2026-07-23 19:06:53.969093932 +0000 UTC m=+356.871680011"
Jul 23 19:06:54.935138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:54.935099    2563 generic.go:356] "Generic (PLEG): container finished" podID="d503566d-7b8f-410f-93ff-2a68a171c406" containerID="153f43ca60b28baef2f893dd6459bddbe5e72ce38858b1e9569a45b19eb5396b" exitCode=0
Jul 23 19:06:54.935559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:54.935193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm" event={"ID":"d503566d-7b8f-410f-93ff-2a68a171c406","Type":"ContainerDied","Data":"153f43ca60b28baef2f893dd6459bddbe5e72ce38858b1e9569a45b19eb5396b"}
Jul 23 19:06:56.066929 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.066907    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:56.145937 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.145904    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle\") pod \"d503566d-7b8f-410f-93ff-2a68a171c406\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") "
Jul 23 19:06:56.145937 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.145944    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5lwg\" (UniqueName: \"kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg\") pod \"d503566d-7b8f-410f-93ff-2a68a171c406\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") "
Jul 23 19:06:56.146132 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.145998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util\") pod \"d503566d-7b8f-410f-93ff-2a68a171c406\" (UID: \"d503566d-7b8f-410f-93ff-2a68a171c406\") "
Jul 23 19:06:56.146347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.146318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle" (OuterVolumeSpecName: "bundle") pod "d503566d-7b8f-410f-93ff-2a68a171c406" (UID: "d503566d-7b8f-410f-93ff-2a68a171c406"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:06:56.148017 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.147993    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg" (OuterVolumeSpecName: "kube-api-access-s5lwg") pod "d503566d-7b8f-410f-93ff-2a68a171c406" (UID: "d503566d-7b8f-410f-93ff-2a68a171c406"). InnerVolumeSpecName "kube-api-access-s5lwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:06:56.150848 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.150827    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util" (OuterVolumeSpecName: "util") pod "d503566d-7b8f-410f-93ff-2a68a171c406" (UID: "d503566d-7b8f-410f-93ff-2a68a171c406"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:06:56.247021 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.246928    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:56.247021 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.246969    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5lwg\" (UniqueName: \"kubernetes.io/projected/d503566d-7b8f-410f-93ff-2a68a171c406-kube-api-access-s5lwg\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:56.247021 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.246980    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503566d-7b8f-410f-93ff-2a68a171c406-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:06:56.944172 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.944144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm" event={"ID":"d503566d-7b8f-410f-93ff-2a68a171c406","Type":"ContainerDied","Data":"4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa"}
Jul 23 19:06:56.944312 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.944185    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmd8bm"
Jul 23 19:06:56.944312 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:56.944188    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b4679c8f099a36f65ae0c76387c8ae2987ce4890acdb040450e1755fc0638fa"
Jul 23 19:06:59.937178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:06:59.937146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w7gp9"
Jul 23 19:07:07.394607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394574    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"]
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394837    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="util"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394849    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="util"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394856    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="pull"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="pull"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394879    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="extract"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394884    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="extract"
Jul 23 19:07:07.395163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.394923    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d503566d-7b8f-410f-93ff-2a68a171c406" containerName="extract"
Jul 23 19:07:07.423399 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.423372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"]
Jul 23 19:07:07.423570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.423471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.426722 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.426695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g4scf\""
Jul 23 19:07:07.427030 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.427009    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 19:07:07.431087 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.431071    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 19:07:07.530997 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.530967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.531151 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.531006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-647jn\" (UniqueName: \"kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.531151 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.531064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.632350 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.632307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.632544 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.632365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.632544 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.632396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-647jn\" (UniqueName: \"kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.632748 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.632726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.632806 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.632744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.642203 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.642177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-647jn\" (UniqueName: \"kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.732562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.732485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:07.893472 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.893443    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"]
Jul 23 19:07:07.895790 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:07.895751    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2dcb601_5c67_43b7_823c_0b86dc32cf67.slice/crio-11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564 WatchSource:0}: Error finding container 11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564: Status 404 returned error can't find the container with id 11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564
Jul 23 19:07:07.984315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:07.984250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv" event={"ID":"d2dcb601-5c67-43b7-823c-0b86dc32cf67","Type":"ContainerStarted","Data":"11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564"}
Jul 23 19:07:08.963298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:08.963224    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8dl2w"]
Jul 23 19:07:08.990296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:08.990263    2563 generic.go:356] "Generic (PLEG): container finished" podID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerID="718f491fd63c58def97d4d80f78812eac22906c04418a794d8ded08816019abe" exitCode=0
Jul 23 19:07:08.997204 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:08.997181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv" event={"ID":"d2dcb601-5c67-43b7-823c-0b86dc32cf67","Type":"ContainerDied","Data":"718f491fd63c58def97d4d80f78812eac22906c04418a794d8ded08816019abe"}
Jul 23 19:07:08.997298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:08.997219    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8dl2w"]
Jul 23 19:07:08.997336 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:08.997299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.001483 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.001467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lrbdl\""
Jul 23 19:07:09.042193 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.042161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99c9x\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-kube-api-access-99c9x\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.042193 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.042191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.143587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.143554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99c9x\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-kube-api-access-99c9x\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.143737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.143588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.152865 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.152831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.152865 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.152850    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99c9x\" (UniqueName: \"kubernetes.io/projected/40df56a5-d84d-4fb8-b436-c5f2a078d01c-kube-api-access-99c9x\") pod \"cert-manager-55b4d954d5-8dl2w\" (UID: \"40df56a5-d84d-4fb8-b436-c5f2a078d01c\") " pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.307132 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.307093    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8dl2w"
Jul 23 19:07:09.463626 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.463590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8dl2w"]
Jul 23 19:07:09.468880 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:09.468854    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40df56a5_d84d_4fb8_b436_c5f2a078d01c.slice/crio-ad7b6b69c75dede07b04b27bac28af142607d7d660dd5ce201f8dcbd70066cb3 WatchSource:0}: Error finding container ad7b6b69c75dede07b04b27bac28af142607d7d660dd5ce201f8dcbd70066cb3: Status 404 returned error can't find the container with id ad7b6b69c75dede07b04b27bac28af142607d7d660dd5ce201f8dcbd70066cb3
Jul 23 19:07:09.995430 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.995348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8dl2w" event={"ID":"40df56a5-d84d-4fb8-b436-c5f2a078d01c","Type":"ContainerStarted","Data":"a31ce1ac1844a2008dbfcbefe34d5b4d484a9e09b9fbb9520a95e0557310b27e"}
Jul 23 19:07:09.995430 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:09.995386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8dl2w" event={"ID":"40df56a5-d84d-4fb8-b436-c5f2a078d01c","Type":"ContainerStarted","Data":"ad7b6b69c75dede07b04b27bac28af142607d7d660dd5ce201f8dcbd70066cb3"}
Jul 23 19:07:10.017859 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:10.017788    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8dl2w" podStartSLOduration=2.017774523 podStartE2EDuration="2.017774523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:10.017242101 +0000 UTC m=+372.919828179" watchObservedRunningTime="2026-07-23 19:07:10.017774523 +0000 UTC m=+372.920360600"
Jul 23 19:07:11.000943 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:11.000910    2563 generic.go:356] "Generic (PLEG): container finished" podID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerID="aab88c4c591edcf7e210f94f4b059b382e9babc1faf5befc157800dc9b871479" exitCode=0
Jul 23 19:07:11.001404 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:11.001000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv" event={"ID":"d2dcb601-5c67-43b7-823c-0b86dc32cf67","Type":"ContainerDied","Data":"aab88c4c591edcf7e210f94f4b059b382e9babc1faf5befc157800dc9b871479"}
Jul 23 19:07:12.005975 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:12.005942    2563 generic.go:356] "Generic (PLEG): container finished" podID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerID="f8e64bda1875ac568dd516ca42e02bfb0f848121d7a9c110d152dc93b61ffede" exitCode=0
Jul 23 19:07:12.006339 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:12.006017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv" event={"ID":"d2dcb601-5c67-43b7-823c-0b86dc32cf67","Type":"ContainerDied","Data":"f8e64bda1875ac568dd516ca42e02bfb0f848121d7a9c110d152dc93b61ffede"}
Jul 23 19:07:13.132438 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.132415    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:13.172946 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.172916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle\") pod \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") "
Jul 23 19:07:13.173095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.172959    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util\") pod \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") "
Jul 23 19:07:13.173095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.172988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-647jn\" (UniqueName: \"kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn\") pod \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\" (UID: \"d2dcb601-5c67-43b7-823c-0b86dc32cf67\") "
Jul 23 19:07:13.173717 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.173691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle" (OuterVolumeSpecName: "bundle") pod "d2dcb601-5c67-43b7-823c-0b86dc32cf67" (UID: "d2dcb601-5c67-43b7-823c-0b86dc32cf67"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:07:13.175064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.175036    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn" (OuterVolumeSpecName: "kube-api-access-647jn") pod "d2dcb601-5c67-43b7-823c-0b86dc32cf67" (UID: "d2dcb601-5c67-43b7-823c-0b86dc32cf67"). InnerVolumeSpecName "kube-api-access-647jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:07:13.273669 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.273590    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:07:13.273669 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.273619    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-647jn\" (UniqueName: \"kubernetes.io/projected/d2dcb601-5c67-43b7-823c-0b86dc32cf67-kube-api-access-647jn\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:07:13.280268 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.280239    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util" (OuterVolumeSpecName: "util") pod "d2dcb601-5c67-43b7-823c-0b86dc32cf67" (UID: "d2dcb601-5c67-43b7-823c-0b86dc32cf67"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:07:13.374656 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:13.374625    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2dcb601-5c67-43b7-823c-0b86dc32cf67-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:07:14.015738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:14.015705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv" event={"ID":"d2dcb601-5c67-43b7-823c-0b86dc32cf67","Type":"ContainerDied","Data":"11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564"}
Jul 23 19:07:14.015895 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:14.015759    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11a03765bf9bb55b4a05e170e590764ec017e00ff88946bc0185d0020e3a1564"
Jul 23 19:07:14.015895 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:14.015725    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmp6vv"
Jul 23 19:07:23.613249 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613216    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"]
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613477    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="extract"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613493    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="extract"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="util"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613520    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="util"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613526    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="pull"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613531    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="pull"
Jul 23 19:07:23.613711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.613576    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2dcb601-5c67-43b7-823c-0b86dc32cf67" containerName="extract"
Jul 23 19:07:23.616551 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.616531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.620739 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.620716    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 19:07:23.621206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.621186    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 19:07:23.621309 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.621227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 19:07:23.621701 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.621682    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wh78r\""
Jul 23 19:07:23.623393 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.623022    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 19:07:23.654772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.654749    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"]
Jul 23 19:07:23.749935 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.749898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.750125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.749940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.750125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.750005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.750125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.750035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.750125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.750060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtl7c\" (UniqueName: \"kubernetes.io/projected/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-kube-api-access-xtl7c\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtl7c\" (UniqueName: \"kubernetes.io/projected/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-kube-api-access-xtl7c\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.851468 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.851421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.853607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.853581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.853764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.853749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.853806 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.853780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.864755 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.864704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtl7c\" (UniqueName: \"kubernetes.io/projected/7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0-kube-api-access-xtl7c\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-qhjvh\" (UID: \"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:23.927323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:23.927295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:24.114052 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:24.114024    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbda8d0_bd26_45c6_8b86_cc5e4a5068d0.slice/crio-0030067515f5d338c9fb200e9fdae3915311c8d96a5a34e58cb2623484bf7868 WatchSource:0}: Error finding container 0030067515f5d338c9fb200e9fdae3915311c8d96a5a34e58cb2623484bf7868: Status 404 returned error can't find the container with id 0030067515f5d338c9fb200e9fdae3915311c8d96a5a34e58cb2623484bf7868
Jul 23 19:07:24.122281 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:24.122256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"]
Jul 23 19:07:25.062066 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:25.062023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh" event={"ID":"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0","Type":"ContainerStarted","Data":"0030067515f5d338c9fb200e9fdae3915311c8d96a5a34e58cb2623484bf7868"}
Jul 23 19:07:28.076775 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:28.076730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh" event={"ID":"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0","Type":"ContainerStarted","Data":"656eeecc866b657597aefe7ce38f039c8dc9374161db6a545d62e16147865cfb"}
Jul 23 19:07:29.082503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:29.082467    2563 generic.go:356] "Generic (PLEG): container finished" podID="7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0" containerID="656eeecc866b657597aefe7ce38f039c8dc9374161db6a545d62e16147865cfb" exitCode=0
Jul 23 19:07:29.082931 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:29.082550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh" event={"ID":"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0","Type":"ContainerDied","Data":"656eeecc866b657597aefe7ce38f039c8dc9374161db6a545d62e16147865cfb"}
Jul 23 19:07:30.088542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:30.088500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh" event={"ID":"7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0","Type":"ContainerStarted","Data":"5624e2cc5f681eac73f11df91b6f8b84651f7cb3e539a2035fece7416629090a"}
Jul 23 19:07:30.088982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:30.088627    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:30.129148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:30.129077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh" podStartSLOduration=2.342950054 podStartE2EDuration="7.129035864s" totalImagesPullingTime="3.700078538s" totalInitContainerRuntime="1.086007272s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:30.127334022 +0000 UTC m=+393.029920131" watchObservedRunningTime="2026-07-23 19:07:30.129035864 +0000 UTC m=+393.031621941"
Jul 23 19:07:41.095169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:41.095137    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-qhjvh"
Jul 23 19:07:44.750992 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.750944    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"]
Jul 23 19:07:44.758960 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.758937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:44.761537 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.761516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 19:07:44.762305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.762287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2kqgr\""
Jul 23 19:07:44.762402 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.762304    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 19:07:44.762402 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.762313    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 19:07:44.762402 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.762294    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 19:07:44.776092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.776067    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"]
Jul 23 19:07:44.920125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.920081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b628d55-fa54-405a-a062-2c2498d640d6-tmp\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:44.920323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.920141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djdq\" (UniqueName: \"kubernetes.io/projected/8b628d55-fa54-405a-a062-2c2498d640d6-kube-api-access-7djdq\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:44.920323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:44.920239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.021356 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.021274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.021356 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.021318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b628d55-fa54-405a-a062-2c2498d640d6-tmp\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.021587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.021358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djdq\" (UniqueName: \"kubernetes.io/projected/8b628d55-fa54-405a-a062-2c2498d640d6-kube-api-access-7djdq\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.021587 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:07:45.021440    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 23 19:07:45.021587 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:07:45.021519    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs podName:8b628d55-fa54-405a-a062-2c2498d640d6 nodeName:}" failed. No retries permitted until 2026-07-23 19:07:45.521498442 +0000 UTC m=+408.424084511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs") pod "kube-auth-proxy-74b646958c-5gfz4" (UID: "8b628d55-fa54-405a-a062-2c2498d640d6") : secret "kube-auth-proxy-tls" not found
Jul 23 19:07:45.023699 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.023676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b628d55-fa54-405a-a062-2c2498d640d6-tmp\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.043057 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.043024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-z8ql5"]
Jul 23 19:07:45.046283 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.046268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.049789 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.049769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 23 19:07:45.049940 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.049797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5r2jq\""
Jul 23 19:07:45.062364 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.062337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djdq\" (UniqueName: \"kubernetes.io/projected/8b628d55-fa54-405a-a062-2c2498d640d6-kube-api-access-7djdq\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.068156 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.068129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-z8ql5"]
Jul 23 19:07:45.222440 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.222406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a7bfef-a3e8-485f-a008-14fcbe7d4988-cert\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.222662 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.222483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ht6h\" (UniqueName: \"kubernetes.io/projected/38a7bfef-a3e8-485f-a008-14fcbe7d4988-kube-api-access-8ht6h\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.323647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.323556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ht6h\" (UniqueName: \"kubernetes.io/projected/38a7bfef-a3e8-485f-a008-14fcbe7d4988-kube-api-access-8ht6h\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.323647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.323628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a7bfef-a3e8-485f-a008-14fcbe7d4988-cert\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.326183 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.326157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38a7bfef-a3e8-485f-a008-14fcbe7d4988-cert\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.333106 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.333085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ht6h\" (UniqueName: \"kubernetes.io/projected/38a7bfef-a3e8-485f-a008-14fcbe7d4988-kube-api-access-8ht6h\") pod \"odh-model-controller-7998c67dc4-z8ql5\" (UID: \"38a7bfef-a3e8-485f-a008-14fcbe7d4988\") " pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.356170 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.356135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:45.500869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.500843    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-z8ql5"]
Jul 23 19:07:45.502795 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:45.502768    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a7bfef_a3e8_485f_a008_14fcbe7d4988.slice/crio-c9670555aaa86eb1ee11838fcfb0e8559bb7f097c59cbe289f227adbdac289d0 WatchSource:0}: Error finding container c9670555aaa86eb1ee11838fcfb0e8559bb7f097c59cbe289f227adbdac289d0: Status 404 returned error can't find the container with id c9670555aaa86eb1ee11838fcfb0e8559bb7f097c59cbe289f227adbdac289d0
Jul 23 19:07:45.525181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.525153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.527584 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.527556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b628d55-fa54-405a-a062-2c2498d640d6-tls-certs\") pod \"kube-auth-proxy-74b646958c-5gfz4\" (UID: \"8b628d55-fa54-405a-a062-2c2498d640d6\") " pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.668090 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.667999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"
Jul 23 19:07:45.807302 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:45.807276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74b646958c-5gfz4"]
Jul 23 19:07:45.809540 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:45.809514    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b628d55_fa54_405a_a062_2c2498d640d6.slice/crio-82f61f43679c79ea3f4e90ccb2c9cfe6c5f79bc76903a40c418ab29acc87fd61 WatchSource:0}: Error finding container 82f61f43679c79ea3f4e90ccb2c9cfe6c5f79bc76903a40c418ab29acc87fd61: Status 404 returned error can't find the container with id 82f61f43679c79ea3f4e90ccb2c9cfe6c5f79bc76903a40c418ab29acc87fd61
Jul 23 19:07:46.090761 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.090730    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7z4q8"]
Jul 23 19:07:46.095356 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.095290    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.097609 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.097581    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 19:07:46.110212 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.108780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7z4q8"]
Jul 23 19:07:46.155375 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.155331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4" event={"ID":"8b628d55-fa54-405a-a062-2c2498d640d6","Type":"ContainerStarted","Data":"82f61f43679c79ea3f4e90ccb2c9cfe6c5f79bc76903a40c418ab29acc87fd61"}
Jul 23 19:07:46.156610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.156584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5" event={"ID":"38a7bfef-a3e8-485f-a008-14fcbe7d4988","Type":"ContainerStarted","Data":"c9670555aaa86eb1ee11838fcfb0e8559bb7f097c59cbe289f227adbdac289d0"}
Jul 23 19:07:46.223418 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.223378    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:07:46.226757 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.226730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.231440 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.231406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c65828c-5904-4308-a58f-92078ea2d3d7-tls-certs\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.231575 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.231516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5qd\" (UniqueName: \"kubernetes.io/projected/3c65828c-5904-4308-a58f-92078ea2d3d7-kube-api-access-lv5qd\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.235182 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.235155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pn7xv\""
Jul 23 19:07:46.235296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.235212    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 19:07:46.235500 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.235466    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 19:07:46.235626 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.235468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 19:07:46.235980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.235960    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 19:07:46.250402 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.250327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:07:46.332454 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c65828c-5904-4308-a58f-92078ea2d3d7-tls-certs\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.332454 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mp75\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.332692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.332692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5qd\" (UniqueName: \"kubernetes.io/projected/3c65828c-5904-4308-a58f-92078ea2d3d7-kube-api-access-lv5qd\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.335644 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.335616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c65828c-5904-4308-a58f-92078ea2d3d7-tls-certs\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.349060 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.348983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5qd\" (UniqueName: \"kubernetes.io/projected/3c65828c-5904-4308-a58f-92078ea2d3d7-kube-api-access-lv5qd\") pod \"model-serving-api-54d89bd5fc-7z4q8\" (UID: \"3c65828c-5904-4308-a58f-92078ea2d3d7\") " pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.410925 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.410887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:46.433435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mp75\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433790 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433580    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.433790 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.433607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.434527 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.434366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.436657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.436630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.437163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.437138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.437312 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.437271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.437712 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.437692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.457705 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.457646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.458591 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.458566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mp75\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75\") pod \"istiod-openshift-gateway-7574459656-qs9v9\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.539108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.538767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:46.607612 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.607581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7z4q8"]
Jul 23 19:07:46.611892 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:46.611846    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c65828c_5904_4308_a58f_92078ea2d3d7.slice/crio-b881011ce4d8e764d45b2fdf96e087f2f4294e5900db52bc4bebc3ca7986f643 WatchSource:0}: Error finding container b881011ce4d8e764d45b2fdf96e087f2f4294e5900db52bc4bebc3ca7986f643: Status 404 returned error can't find the container with id b881011ce4d8e764d45b2fdf96e087f2f4294e5900db52bc4bebc3ca7986f643
Jul 23 19:07:46.747045 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:46.746924    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:07:46.752562 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:46.752448    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b9bc52_1549_4c4a_98d5_284cda10dcbc.slice/crio-7f778cb167a2b0976c911c0234d0f2497d9d585b3bf4c430e0724814b07a188c WatchSource:0}: Error finding container 7f778cb167a2b0976c911c0234d0f2497d9d585b3bf4c430e0724814b07a188c: Status 404 returned error can't find the container with id 7f778cb167a2b0976c911c0234d0f2497d9d585b3bf4c430e0724814b07a188c
Jul 23 19:07:47.162618 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:47.162586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" event={"ID":"14b9bc52-1549-4c4a-98d5-284cda10dcbc","Type":"ContainerStarted","Data":"7f778cb167a2b0976c911c0234d0f2497d9d585b3bf4c430e0724814b07a188c"}
Jul 23 19:07:47.163644 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:47.163623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8" event={"ID":"3c65828c-5904-4308-a58f-92078ea2d3d7","Type":"ContainerStarted","Data":"b881011ce4d8e764d45b2fdf96e087f2f4294e5900db52bc4bebc3ca7986f643"}
Jul 23 19:07:50.994032 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:50.993982    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 23 19:07:50.994429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:50.994075    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 23 19:07:51.183860 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.183740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4" event={"ID":"8b628d55-fa54-405a-a062-2c2498d640d6","Type":"ContainerStarted","Data":"2101ddbe3c1af6e8626a0b77c336dcdc4c3589ca413f74dc75bd855e10309f36"}
Jul 23 19:07:51.185507 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.185467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" event={"ID":"14b9bc52-1549-4c4a-98d5-284cda10dcbc","Type":"ContainerStarted","Data":"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"}
Jul 23 19:07:51.185642 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.185564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:51.186906 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.186884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5" event={"ID":"38a7bfef-a3e8-485f-a008-14fcbe7d4988","Type":"ContainerStarted","Data":"620497a1805d93530594f8a796e7506384619cdd7e89f702f9c97d6753fea054"}
Jul 23 19:07:51.187050 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.187034    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:07:51.188363 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.188342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8" event={"ID":"3c65828c-5904-4308-a58f-92078ea2d3d7","Type":"ContainerStarted","Data":"c2bff758aa56c32ea4dd7a771e88cc0f28aaceaf94fe0065bc9ae78a81ca4f89"}
Jul 23 19:07:51.188466 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.188407    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:07:51.213152 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.213089    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74b646958c-5gfz4" podStartSLOduration=2.112775096 podStartE2EDuration="7.21306887s" totalImagesPullingTime="5.100293774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:51.211239765 +0000 UTC m=+414.113825853" watchObservedRunningTime="2026-07-23 19:07:51.21306887 +0000 UTC m=+414.115654949"
Jul 23 19:07:51.250323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.250263    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5" podStartSLOduration=0.843431258 podStartE2EDuration="6.250242004s" totalImagesPullingTime="5.406810746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:51.249940628 +0000 UTC m=+414.152526705" watchObservedRunningTime="2026-07-23 19:07:51.250242004 +0000 UTC m=+414.152828085"
Jul 23 19:07:51.284333 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.284269    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" podStartSLOduration=1.046127615 podStartE2EDuration="5.284248319s" totalImagesPullingTime="4.238120704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:51.282903774 +0000 UTC m=+414.185489876" watchObservedRunningTime="2026-07-23 19:07:51.284248319 +0000 UTC m=+414.186834396"
Jul 23 19:07:51.304924 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:51.304868    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8" podStartSLOduration=1.009848072 podStartE2EDuration="5.304849193s" totalImagesPullingTime="4.295001121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:51.303696719 +0000 UTC m=+414.206282798" watchObservedRunningTime="2026-07-23 19:07:51.304849193 +0000 UTC m=+414.207435270"
Jul 23 19:07:52.193888 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:52.193854    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qs9v9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 19:07:52.194252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:52.193902    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 19:07:55.193092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:55.193057    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:07:57.752482 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.752449    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"]
Jul 23 19:07:57.757278 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.757259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.774380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.774351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 23 19:07:57.774380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.774363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-86lbg\""
Jul 23 19:07:57.776763 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.776741    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"]
Jul 23 19:07:57.779899 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.779884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.823908 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.823878    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wdtxr\""
Jul 23 19:07:57.828300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.828274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2jwf\" (UniqueName: \"kubernetes.io/projected/55e15c55-30aa-4801-a6c0-d3af4661000c-kube-api-access-m2jwf\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.828401 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.828319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgbqq\" (UniqueName: \"kubernetes.io/projected/dd84eb65-3b47-4676-bcbe-05c787b12797-kube-api-access-dgbqq\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.828401 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.828352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e15c55-30aa-4801-a6c0-d3af4661000c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.828490 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.828467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd84eb65-3b47-4676-bcbe-05c787b12797-cert\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.849267 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.849238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"]
Jul 23 19:07:57.849455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.849441    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 19:07:57.864527 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.864502    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"]
Jul 23 19:07:57.929277 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.929248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgbqq\" (UniqueName: \"kubernetes.io/projected/dd84eb65-3b47-4676-bcbe-05c787b12797-kube-api-access-dgbqq\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.929457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.929285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e15c55-30aa-4801-a6c0-d3af4661000c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.929457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.929339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd84eb65-3b47-4676-bcbe-05c787b12797-cert\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.929457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.929361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2jwf\" (UniqueName: \"kubernetes.io/projected/55e15c55-30aa-4801-a6c0-d3af4661000c-kube-api-access-m2jwf\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.931840 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.931797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd84eb65-3b47-4676-bcbe-05c787b12797-cert\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.931945 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.931857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e15c55-30aa-4801-a6c0-d3af4661000c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:57.980305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.980275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgbqq\" (UniqueName: \"kubernetes.io/projected/dd84eb65-3b47-4676-bcbe-05c787b12797-kube-api-access-dgbqq\") pod \"kserve-controller-manager-d6f4b65b8-75zkd\" (UID: \"dd84eb65-3b47-4676-bcbe-05c787b12797\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:57.994900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:57.994874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2jwf\" (UniqueName: \"kubernetes.io/projected/55e15c55-30aa-4801-a6c0-d3af4661000c-kube-api-access-m2jwf\") pod \"llmisvc-controller-manager-777c6bd7d5-nvx42\" (UID: \"55e15c55-30aa-4801-a6c0-d3af4661000c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:58.067937 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:58.067836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:07:58.089155 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:58.089130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:07:58.258020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:58.257993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"]
Jul 23 19:07:58.259509 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:58.259481    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd84eb65_3b47_4676_bcbe_05c787b12797.slice/crio-24fc818fcb62d8a8a8815cb4e60ec56da60142cd709ebf91f856b9916b92ae00 WatchSource:0}: Error finding container 24fc818fcb62d8a8a8815cb4e60ec56da60142cd709ebf91f856b9916b92ae00: Status 404 returned error can't find the container with id 24fc818fcb62d8a8a8815cb4e60ec56da60142cd709ebf91f856b9916b92ae00
Jul 23 19:07:58.263513 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:58.263489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"]
Jul 23 19:07:58.266020 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:07:58.265995    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod55e15c55_30aa_4801_a6c0_d3af4661000c.slice/crio-c12690b9706f1f0d8895006062e233b8cfd68e7ee3b33f757f59df7033a17a8d WatchSource:0}: Error finding container c12690b9706f1f0d8895006062e233b8cfd68e7ee3b33f757f59df7033a17a8d: Status 404 returned error can't find the container with id c12690b9706f1f0d8895006062e233b8cfd68e7ee3b33f757f59df7033a17a8d
Jul 23 19:07:59.226080 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:59.226041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42" event={"ID":"55e15c55-30aa-4801-a6c0-d3af4661000c","Type":"ContainerStarted","Data":"c12690b9706f1f0d8895006062e233b8cfd68e7ee3b33f757f59df7033a17a8d"}
Jul 23 19:07:59.227235 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:07:59.227211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd" event={"ID":"dd84eb65-3b47-4676-bcbe-05c787b12797","Type":"ContainerStarted","Data":"24fc818fcb62d8a8a8815cb4e60ec56da60142cd709ebf91f856b9916b92ae00"}
Jul 23 19:08:02.196250 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:02.196222    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7z4q8"
Jul 23 19:08:02.209446 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:02.209415    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-z8ql5"
Jul 23 19:08:02.262782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:02.262105    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd" event={"ID":"dd84eb65-3b47-4676-bcbe-05c787b12797","Type":"ContainerStarted","Data":"71264625c8a41868cb394f026778b1c5b65f3c319cb8cf0f25833dd5cd98825c"}
Jul 23 19:08:02.262782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:02.262198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:08:02.285674 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:02.285614    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd" podStartSLOduration=2.057912259 podStartE2EDuration="5.285590917s" totalImagesPullingTime="3.227678658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:08:02.284443018 +0000 UTC m=+425.187029124" watchObservedRunningTime="2026-07-23 19:08:02.285590917 +0000 UTC m=+425.188176996"
Jul 23 19:08:03.268070 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:03.268031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42" event={"ID":"55e15c55-30aa-4801-a6c0-d3af4661000c","Type":"ContainerStarted","Data":"29c96222d4f162061c788b2a4958760ca2bada5bded79beb26a830c4b9c47673"}
Jul 23 19:08:03.268480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:03.268236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:08:03.292045 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:03.291992    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42" podStartSLOduration=2.432033706 podStartE2EDuration="6.291976694s" totalImagesPullingTime="3.859942988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:08:03.291204 +0000 UTC m=+426.193790077" watchObservedRunningTime="2026-07-23 19:08:03.291976694 +0000 UTC m=+426.194562797"
Jul 23 19:08:05.910146 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:05.910114    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:08:05.910564 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:05.910293    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerName="discovery" containerID="cri-o://f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42" gracePeriod=30
Jul 23 19:08:06.163300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.163237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:08:06.190211 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190245    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190281    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190334    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mp75\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190378    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190635 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap\") pod \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\" (UID: \"14b9bc52-1549-4c4a-98d5-284cda10dcbc\") "
Jul 23 19:08:06.190967 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.190935    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:08:06.192831 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.192782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:08:06.192994 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.192971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts" (OuterVolumeSpecName: "cacerts") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:08:06.193071 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.193016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs" (OuterVolumeSpecName: "local-certs") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:08:06.193195 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.193167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75" (OuterVolumeSpecName: "kube-api-access-2mp75") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "kube-api-access-2mp75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:08:06.193260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.193238    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:08:06.193540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.193495    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "14b9bc52-1549-4c4a-98d5-284cda10dcbc" (UID: "14b9bc52-1549-4c4a-98d5-284cda10dcbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:08:06.281784 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.281747    2563 generic.go:356] "Generic (PLEG): container finished" podID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerID="f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42" exitCode=0
Jul 23 19:08:06.281980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.281793    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"
Jul 23 19:08:06.281980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.281839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" event={"ID":"14b9bc52-1549-4c4a-98d5-284cda10dcbc","Type":"ContainerDied","Data":"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"}
Jul 23 19:08:06.281980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.281891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9" event={"ID":"14b9bc52-1549-4c4a-98d5-284cda10dcbc","Type":"ContainerDied","Data":"7f778cb167a2b0976c911c0234d0f2497d9d585b3bf4c430e0724814b07a188c"}
Jul 23 19:08:06.281980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.281910    2563 scope.go:117] "RemoveContainer" containerID="f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"
Jul 23 19:08:06.291114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291058    2563 scope.go:117] "RemoveContainer" containerID="f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"
Jul 23 19:08:06.291411 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291392    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-ca-configmap\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291411 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:08:06.291387    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42\": container with ID starting with f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42 not found: ID does not exist" containerID="f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"
Jul 23 19:08:06.291572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291460    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42"} err="failed to get container status \"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42\": rpc error: code = NotFound desc = could not find container \"f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42\": container with ID starting with f3bdfbfe0e2d034e8956015b522435b2893b142f4ed74fb84fc747b6f6b18a42 not found: ID does not exist"
Jul 23 19:08:06.291572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291414    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-cacerts\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291529    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-csr-dns-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291547    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-kubeconfig\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291561    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b9bc52-1549-4c4a-98d5-284cda10dcbc-local-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291573    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mp75\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-kube-api-access-2mp75\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.291742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.291592    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b9bc52-1549-4c4a-98d5-284cda10dcbc-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:06.313569 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.311603    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:08:06.319351 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:06.319313    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qs9v9"]
Jul 23 19:08:07.522339 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:07.522299    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" path="/var/lib/kubelet/pods/14b9bc52-1549-4c4a-98d5-284cda10dcbc/volumes"
Jul 23 19:08:33.273018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:33.272987    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-75zkd"
Jul 23 19:08:34.275888 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:34.275862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nvx42"
Jul 23 19:08:45.555585 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.555549    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:45.556108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.555892    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerName="discovery"
Jul 23 19:08:45.556108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.555906    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerName="discovery"
Jul 23 19:08:45.556108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.555969    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b9bc52-1549-4c4a-98d5-284cda10dcbc" containerName="discovery"
Jul 23 19:08:45.565993 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.565966    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:45.568681 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.568660    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 19:08:45.569592 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.569555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 19:08:45.569694 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.569564    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8pd68\""
Jul 23 19:08:45.581869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.581843    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:45.703013 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.702975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llzf5\" (UniqueName: \"kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5\") pod \"kuadrant-operator-catalog-kdpgg\" (UID: \"65cc9683-cb8c-4c83-a519-fb2361fd2d59\") " pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:45.804144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.804113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzf5\" (UniqueName: \"kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5\") pod \"kuadrant-operator-catalog-kdpgg\" (UID: \"65cc9683-cb8c-4c83-a519-fb2361fd2d59\") " pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:45.815755 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.815686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llzf5\" (UniqueName: \"kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5\") pod \"kuadrant-operator-catalog-kdpgg\" (UID: \"65cc9683-cb8c-4c83-a519-fb2361fd2d59\") " pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:45.876001 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.875969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:45.906194 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:45.906162    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:46.030038 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.030010    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:46.031175 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:46.031152    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cc9683_cb8c_4c83_a519_fb2361fd2d59.slice/crio-c9778dafb70f402e9dd107d26ae081fda8502d0a48fabeeffb730fca07d57021 WatchSource:0}: Error finding container c9778dafb70f402e9dd107d26ae081fda8502d0a48fabeeffb730fca07d57021: Status 404 returned error can't find the container with id c9778dafb70f402e9dd107d26ae081fda8502d0a48fabeeffb730fca07d57021
Jul 23 19:08:46.118148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.118058    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bdlbn"]
Jul 23 19:08:46.122635 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.122610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:46.143462 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.143433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bdlbn"]
Jul 23 19:08:46.207290 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.207250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9bvr\" (UniqueName: \"kubernetes.io/projected/41079020-1b5d-4d01-bf75-dede8f45852e-kube-api-access-m9bvr\") pod \"kuadrant-operator-catalog-bdlbn\" (UID: \"41079020-1b5d-4d01-bf75-dede8f45852e\") " pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:46.308140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.308111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9bvr\" (UniqueName: \"kubernetes.io/projected/41079020-1b5d-4d01-bf75-dede8f45852e-kube-api-access-m9bvr\") pod \"kuadrant-operator-catalog-bdlbn\" (UID: \"41079020-1b5d-4d01-bf75-dede8f45852e\") " pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:46.326482 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.326457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9bvr\" (UniqueName: \"kubernetes.io/projected/41079020-1b5d-4d01-bf75-dede8f45852e-kube-api-access-m9bvr\") pod \"kuadrant-operator-catalog-bdlbn\" (UID: \"41079020-1b5d-4d01-bf75-dede8f45852e\") " pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:46.431646 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.431563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:46.456047 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.456007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" event={"ID":"65cc9683-cb8c-4c83-a519-fb2361fd2d59","Type":"ContainerStarted","Data":"c9778dafb70f402e9dd107d26ae081fda8502d0a48fabeeffb730fca07d57021"}
Jul 23 19:08:46.567776 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:46.567746    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bdlbn"]
Jul 23 19:08:46.569753 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:46.569725    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41079020_1b5d_4d01_bf75_dede8f45852e.slice/crio-75948b30700d53e3adf7d96d18f45003be8c5cc97412bfbdc813897555ca4332 WatchSource:0}: Error finding container 75948b30700d53e3adf7d96d18f45003be8c5cc97412bfbdc813897555ca4332: Status 404 returned error can't find the container with id 75948b30700d53e3adf7d96d18f45003be8c5cc97412bfbdc813897555ca4332
Jul 23 19:08:47.463129 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:47.463097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn" event={"ID":"41079020-1b5d-4d01-bf75-dede8f45852e","Type":"ContainerStarted","Data":"75948b30700d53e3adf7d96d18f45003be8c5cc97412bfbdc813897555ca4332"}
Jul 23 19:08:48.469296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.469194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" event={"ID":"65cc9683-cb8c-4c83-a519-fb2361fd2d59","Type":"ContainerStarted","Data":"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"}
Jul 23 19:08:48.469296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.469220    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" podUID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" containerName="registry-server" containerID="cri-o://c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8" gracePeriod=2
Jul 23 19:08:48.470721 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.470691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn" event={"ID":"41079020-1b5d-4d01-bf75-dede8f45852e","Type":"ContainerStarted","Data":"013442a644fa7558f524f76a24b08fb45909b1cdf57dae70d3e7daf88da17169"}
Jul 23 19:08:48.513352 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.513301    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" podStartSLOduration=1.508243676 podStartE2EDuration="3.513284953s" totalImagesPullingTime="2.005041277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:08:48.512120461 +0000 UTC m=+471.414706538" watchObservedRunningTime="2026-07-23 19:08:48.513284953 +0000 UTC m=+471.415871030"
Jul 23 19:08:48.543785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.543737    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn" podStartSLOduration=1.076513039 podStartE2EDuration="2.543721337s" totalImagesPullingTime="1.467208298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:08:48.543219493 +0000 UTC m=+471.445805590" watchObservedRunningTime="2026-07-23 19:08:48.543721337 +0000 UTC m=+471.446307415"
Jul 23 19:08:48.728608 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.728543    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:48.830090 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.830056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llzf5\" (UniqueName: \"kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5\") pod \"65cc9683-cb8c-4c83-a519-fb2361fd2d59\" (UID: \"65cc9683-cb8c-4c83-a519-fb2361fd2d59\") "
Jul 23 19:08:48.832181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.832152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5" (OuterVolumeSpecName: "kube-api-access-llzf5") pod "65cc9683-cb8c-4c83-a519-fb2361fd2d59" (UID: "65cc9683-cb8c-4c83-a519-fb2361fd2d59"). InnerVolumeSpecName "kube-api-access-llzf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:08:48.930845 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:48.930795    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llzf5\" (UniqueName: \"kubernetes.io/projected/65cc9683-cb8c-4c83-a519-fb2361fd2d59-kube-api-access-llzf5\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:08:49.476695 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.476615    2563 generic.go:356] "Generic (PLEG): container finished" podID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" containerID="c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8" exitCode=0
Jul 23 19:08:49.476695 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.476663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg"
Jul 23 19:08:49.477160 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.476699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" event={"ID":"65cc9683-cb8c-4c83-a519-fb2361fd2d59","Type":"ContainerDied","Data":"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"}
Jul 23 19:08:49.477160 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.476742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kdpgg" event={"ID":"65cc9683-cb8c-4c83-a519-fb2361fd2d59","Type":"ContainerDied","Data":"c9778dafb70f402e9dd107d26ae081fda8502d0a48fabeeffb730fca07d57021"}
Jul 23 19:08:49.477160 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.476761    2563 scope.go:117] "RemoveContainer" containerID="c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"
Jul 23 19:08:49.485704 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.485685    2563 scope.go:117] "RemoveContainer" containerID="c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"
Jul 23 19:08:49.485977 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:08:49.485959    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8\": container with ID starting with c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8 not found: ID does not exist" containerID="c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"
Jul 23 19:08:49.486050 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.485986    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8"} err="failed to get container status \"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8\": rpc error: code = NotFound desc = could not find container \"c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8\": container with ID starting with c1f3e409464077c068bf71ad29c5aaba709c2ca15e40b54402f8c011d5de95c8 not found: ID does not exist"
Jul 23 19:08:49.501232 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.501207    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:49.503578 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.503559    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kdpgg"]
Jul 23 19:08:49.521330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:49.521300    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" path="/var/lib/kubelet/pods/65cc9683-cb8c-4c83-a519-fb2361fd2d59/volumes"
Jul 23 19:08:56.431945 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:56.431908    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:56.432557 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:56.431970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:56.454523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:56.454494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:56.529651 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:56.529623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-bdlbn"
Jul 23 19:08:57.543756 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.543723    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"]
Jul 23 19:08:57.544107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.544043    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" containerName="registry-server"
Jul 23 19:08:57.544107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.544056    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" containerName="registry-server"
Jul 23 19:08:57.544180 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.544114    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="65cc9683-cb8c-4c83-a519-fb2361fd2d59" containerName="registry-server"
Jul 23 19:08:57.548494 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.548477    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.550989 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.550968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zpxm7\""
Jul 23 19:08:57.556661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.556637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"]
Jul 23 19:08:57.703044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.702947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fl27\" (UniqueName: \"kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.703044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.703012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.703249 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.703073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.803542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.803500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fl27\" (UniqueName: \"kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.803703 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.803564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.803703 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.803668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.803918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.803898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.803983 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.803938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.813327 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.813295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fl27\" (UniqueName: \"kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.857324 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.857293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:08:57.991644 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:57.991578    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"]
Jul 23 19:08:57.993743 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:57.993717    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94d4809_4657_4f15_9f61_423dfaa0b77f.slice/crio-e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1 WatchSource:0}: Error finding container e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1: Status 404 returned error can't find the container with id e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1
Jul 23 19:08:58.160917 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.160887    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"]
Jul 23 19:08:58.164269 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.164253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.180435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.180405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"]
Jul 23 19:08:58.308715 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.308672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.308715 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.308715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.308959 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.308792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v8kx\" (UniqueName: \"kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.409877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.409838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v8kx\" (UniqueName: \"kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.409991 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.409932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.409991 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.409950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.410296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.410275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.410336 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.410315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.421201 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.421170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v8kx\" (UniqueName: \"kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.473173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.473141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:08:58.518939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.518907    2563 generic.go:356] "Generic (PLEG): container finished" podID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerID="59987798b55ab42bf5e9c7830335ecb2720626c3eb344c15ede26f25258ee7a5" exitCode=0
Jul 23 19:08:58.519128 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.518946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h" event={"ID":"f94d4809-4657-4f15-9f61-423dfaa0b77f","Type":"ContainerDied","Data":"59987798b55ab42bf5e9c7830335ecb2720626c3eb344c15ede26f25258ee7a5"}
Jul 23 19:08:58.519128 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.518982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h" event={"ID":"f94d4809-4657-4f15-9f61-423dfaa0b77f","Type":"ContainerStarted","Data":"e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1"}
Jul 23 19:08:58.615523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.615492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"]
Jul 23 19:08:58.617282 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:58.617258    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c51f2dd_f7c7_40fb_a48d_2cf89d49ee9b.slice/crio-555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199 WatchSource:0}: Error finding container 555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199: Status 404 returned error can't find the container with id 555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199
Jul 23 19:08:58.760347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.760313    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"]
Jul 23 19:08:58.763686 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.763669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:58.773772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.773747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"]
Jul 23 19:08:58.914206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.914172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z84p\" (UniqueName: \"kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:58.914206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.914209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:58.914417 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:58.914301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.015572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.015533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.015730 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.015595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z84p\" (UniqueName: \"kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.015730 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.015622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.015979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.015950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.016031 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.016001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.025301 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.025274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z84p\" (UniqueName: \"kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.073322 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.073237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:08:59.212107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.211900    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"]
Jul 23 19:08:59.214077 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:59.214047    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62aa6d5_b444_4f57_8208_179aa960583e.slice/crio-721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51 WatchSource:0}: Error finding container 721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51: Status 404 returned error can't find the container with id 721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51
Jul 23 19:08:59.341981 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.341910    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"]
Jul 23 19:08:59.345672 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.345649    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.355492 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.355467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"]
Jul 23 19:08:59.518733 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.518707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b76f9\" (UniqueName: \"kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.518879 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.518752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.518879 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.518776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.525675 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.525649    2563 generic.go:356] "Generic (PLEG): container finished" podID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerID="c9e5df1b3ee6de8278ec965fc761cf888bb7a9e62ca26656486465f9442333ed" exitCode=0
Jul 23 19:08:59.525801 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.525724    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h" event={"ID":"f94d4809-4657-4f15-9f61-423dfaa0b77f","Type":"ContainerDied","Data":"c9e5df1b3ee6de8278ec965fc761cf888bb7a9e62ca26656486465f9442333ed"}
Jul 23 19:08:59.527292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.527271    2563 generic.go:356] "Generic (PLEG): container finished" podID="a62aa6d5-b444-4f57-8208-179aa960583e" containerID="ffe66cf1b783e5ad636b85fb759f20d11a770639d61493c91985e83a36b7898c" exitCode=0
Jul 23 19:08:59.527353 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.527319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv" event={"ID":"a62aa6d5-b444-4f57-8208-179aa960583e","Type":"ContainerDied","Data":"ffe66cf1b783e5ad636b85fb759f20d11a770639d61493c91985e83a36b7898c"}
Jul 23 19:08:59.527403 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.527365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv" event={"ID":"a62aa6d5-b444-4f57-8208-179aa960583e","Type":"ContainerStarted","Data":"721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51"}
Jul 23 19:08:59.528864 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.528839    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerID="86bc44a103e2763f1ae342daf41262e26d09e93227e17003844254f93ff67632" exitCode=0
Jul 23 19:08:59.528939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.528914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerDied","Data":"86bc44a103e2763f1ae342daf41262e26d09e93227e17003844254f93ff67632"}
Jul 23 19:08:59.528975 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.528952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerStarted","Data":"555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199"}
Jul 23 19:08:59.619733 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.619704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.620164 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.619743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.620164 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.619861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b76f9\" (UniqueName: \"kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.620268 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.620169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.620268 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.620198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.630551 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.630528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b76f9\" (UniqueName: \"kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.657281 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.657252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:08:59.800923 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:08:59.800708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"]
Jul 23 19:08:59.803514 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:08:59.803476    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97451924_94b7_41ed_960e_d02e1054172c.slice/crio-38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692 WatchSource:0}: Error finding container 38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692: Status 404 returned error can't find the container with id 38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692
Jul 23 19:09:00.535772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.535733    2563 generic.go:356] "Generic (PLEG): container finished" podID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerID="e713ae6dd85405b994bdbb6e6d543372860e6c6cdaf77323573e8dc4445fabe5" exitCode=0
Jul 23 19:09:00.536064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.535824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h" event={"ID":"f94d4809-4657-4f15-9f61-423dfaa0b77f","Type":"ContainerDied","Data":"e713ae6dd85405b994bdbb6e6d543372860e6c6cdaf77323573e8dc4445fabe5"}
Jul 23 19:09:00.538031 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.538009    2563 generic.go:356] "Generic (PLEG): container finished" podID="a62aa6d5-b444-4f57-8208-179aa960583e" containerID="c9a00ee64a2fbe00ff49cdac466f395f40c6a4f687a62bd4875b5be2cea31ca4" exitCode=0
Jul 23 19:09:00.538134 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.538089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv" event={"ID":"a62aa6d5-b444-4f57-8208-179aa960583e","Type":"ContainerDied","Data":"c9a00ee64a2fbe00ff49cdac466f395f40c6a4f687a62bd4875b5be2cea31ca4"}
Jul 23 19:09:00.539993 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.539963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerStarted","Data":"fd7dd8827746ed7725c5601cf8ba8f7a56fb629face7a1c8fcd7986a76a12378"}
Jul 23 19:09:00.541936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.541703    2563 generic.go:356] "Generic (PLEG): container finished" podID="97451924-94b7-41ed-960e-d02e1054172c" containerID="47be9dbe42d55eb278beb2c34e0d837d581076a47a2062ea31a34af21af40d54" exitCode=0
Jul 23 19:09:00.541936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.541737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg" event={"ID":"97451924-94b7-41ed-960e-d02e1054172c","Type":"ContainerDied","Data":"47be9dbe42d55eb278beb2c34e0d837d581076a47a2062ea31a34af21af40d54"}
Jul 23 19:09:00.541936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:00.541766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg" event={"ID":"97451924-94b7-41ed-960e-d02e1054172c","Type":"ContainerStarted","Data":"38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692"}
Jul 23 19:09:01.548742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.548706    2563 generic.go:356] "Generic (PLEG): container finished" podID="a62aa6d5-b444-4f57-8208-179aa960583e" containerID="d06b0404f72355b0fcb684a2f35798ff8b94ceb1dd04b44627ef17e994695ba5" exitCode=0
Jul 23 19:09:01.549177 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.548792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv" event={"ID":"a62aa6d5-b444-4f57-8208-179aa960583e","Type":"ContainerDied","Data":"d06b0404f72355b0fcb684a2f35798ff8b94ceb1dd04b44627ef17e994695ba5"}
Jul 23 19:09:01.550609 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.550585    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerID="fd7dd8827746ed7725c5601cf8ba8f7a56fb629face7a1c8fcd7986a76a12378" exitCode=0
Jul 23 19:09:01.550734 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.550677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerDied","Data":"fd7dd8827746ed7725c5601cf8ba8f7a56fb629face7a1c8fcd7986a76a12378"}
Jul 23 19:09:01.552670 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.552643    2563 generic.go:356] "Generic (PLEG): container finished" podID="97451924-94b7-41ed-960e-d02e1054172c" containerID="0b2157823219832e22d242d1e50c0d6a006be45ea72b886740b1cb59781d4485" exitCode=0
Jul 23 19:09:01.552775 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.552654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg" event={"ID":"97451924-94b7-41ed-960e-d02e1054172c","Type":"ContainerDied","Data":"0b2157823219832e22d242d1e50c0d6a006be45ea72b886740b1cb59781d4485"}
Jul 23 19:09:01.686030 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.685713    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:09:01.838138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.838109    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util\") pod \"f94d4809-4657-4f15-9f61-423dfaa0b77f\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") "
Jul 23 19:09:01.838274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.838159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle\") pod \"f94d4809-4657-4f15-9f61-423dfaa0b77f\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") "
Jul 23 19:09:01.838274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.838189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fl27\" (UniqueName: \"kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27\") pod \"f94d4809-4657-4f15-9f61-423dfaa0b77f\" (UID: \"f94d4809-4657-4f15-9f61-423dfaa0b77f\") "
Jul 23 19:09:01.838686 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.838662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle" (OuterVolumeSpecName: "bundle") pod "f94d4809-4657-4f15-9f61-423dfaa0b77f" (UID: "f94d4809-4657-4f15-9f61-423dfaa0b77f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:01.840281 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.840255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27" (OuterVolumeSpecName: "kube-api-access-9fl27") pod "f94d4809-4657-4f15-9f61-423dfaa0b77f" (UID: "f94d4809-4657-4f15-9f61-423dfaa0b77f"). InnerVolumeSpecName "kube-api-access-9fl27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:01.843175 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.843151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util" (OuterVolumeSpecName: "util") pod "f94d4809-4657-4f15-9f61-423dfaa0b77f" (UID: "f94d4809-4657-4f15-9f61-423dfaa0b77f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:01.939589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.939499    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:01.939589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.939536    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fl27\" (UniqueName: \"kubernetes.io/projected/f94d4809-4657-4f15-9f61-423dfaa0b77f-kube-api-access-9fl27\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:01.939589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:01.939548    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d4809-4657-4f15-9f61-423dfaa0b77f-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:02.559072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.559041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h" event={"ID":"f94d4809-4657-4f15-9f61-423dfaa0b77f","Type":"ContainerDied","Data":"e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1"}
Jul 23 19:09:02.559072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.559062    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h"
Jul 23 19:09:02.559525 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.559092    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4ddaf7c9ce0a37b8c62d35b625179acecad4c6c40e06ac719a6ca659f942bd1"
Jul 23 19:09:02.567230 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.567201    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerID="84fa3ff96c0280452b01545617c2a4fc1fcd829ba7fd32e8b20c4619db858a74" exitCode=0
Jul 23 19:09:02.567397 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.567285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerDied","Data":"84fa3ff96c0280452b01545617c2a4fc1fcd829ba7fd32e8b20c4619db858a74"}
Jul 23 19:09:02.569585 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.569465    2563 generic.go:356] "Generic (PLEG): container finished" podID="97451924-94b7-41ed-960e-d02e1054172c" containerID="1711fff8f1335fc0a08d2c2e1233380939e1ec9459ab5919f7088fd17c9825b5" exitCode=0
Jul 23 19:09:02.569664 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.569580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg" event={"ID":"97451924-94b7-41ed-960e-d02e1054172c","Type":"ContainerDied","Data":"1711fff8f1335fc0a08d2c2e1233380939e1ec9459ab5919f7088fd17c9825b5"}
Jul 23 19:09:02.701808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.701787    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:09:02.846169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.846076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z84p\" (UniqueName: \"kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p\") pod \"a62aa6d5-b444-4f57-8208-179aa960583e\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") "
Jul 23 19:09:02.846169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.846169    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util\") pod \"a62aa6d5-b444-4f57-8208-179aa960583e\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") "
Jul 23 19:09:02.846330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.846226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle\") pod \"a62aa6d5-b444-4f57-8208-179aa960583e\" (UID: \"a62aa6d5-b444-4f57-8208-179aa960583e\") "
Jul 23 19:09:02.846710 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.846678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle" (OuterVolumeSpecName: "bundle") pod "a62aa6d5-b444-4f57-8208-179aa960583e" (UID: "a62aa6d5-b444-4f57-8208-179aa960583e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:02.848275 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.848255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p" (OuterVolumeSpecName: "kube-api-access-7z84p") pod "a62aa6d5-b444-4f57-8208-179aa960583e" (UID: "a62aa6d5-b444-4f57-8208-179aa960583e"). InnerVolumeSpecName "kube-api-access-7z84p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:02.852114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.852088    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util" (OuterVolumeSpecName: "util") pod "a62aa6d5-b444-4f57-8208-179aa960583e" (UID: "a62aa6d5-b444-4f57-8208-179aa960583e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:02.947150 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.947116    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:02.947150 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.947145    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z84p\" (UniqueName: \"kubernetes.io/projected/a62aa6d5-b444-4f57-8208-179aa960583e-kube-api-access-7z84p\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:02.947150 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:02.947155    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a62aa6d5-b444-4f57-8208-179aa960583e-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.575759 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.575736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv"
Jul 23 19:09:03.576155 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.575763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv" event={"ID":"a62aa6d5-b444-4f57-8208-179aa960583e","Type":"ContainerDied","Data":"721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51"}
Jul 23 19:09:03.576155 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.575804    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="721164272cc1ea02464e54f5f34fa79926e112ecb02ed2dcdec480d58f0b2b51"
Jul 23 19:09:03.722782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.722758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:09:03.752378 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.752349    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:09:03.853486 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853417    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle\") pod \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") "
Jul 23 19:09:03.853486 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853477    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util\") pod \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") "
Jul 23 19:09:03.853683 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle\") pod \"97451924-94b7-41ed-960e-d02e1054172c\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") "
Jul 23 19:09:03.853683 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b76f9\" (UniqueName: \"kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9\") pod \"97451924-94b7-41ed-960e-d02e1054172c\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") "
Jul 23 19:09:03.853683 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853575    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v8kx\" (UniqueName: \"kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx\") pod \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\" (UID: \"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b\") "
Jul 23 19:09:03.853683 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.853591    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util\") pod \"97451924-94b7-41ed-960e-d02e1054172c\" (UID: \"97451924-94b7-41ed-960e-d02e1054172c\") "
Jul 23 19:09:03.854179 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.854145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle" (OuterVolumeSpecName: "bundle") pod "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" (UID: "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:03.854313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.854288    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle" (OuterVolumeSpecName: "bundle") pod "97451924-94b7-41ed-960e-d02e1054172c" (UID: "97451924-94b7-41ed-960e-d02e1054172c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:03.855722 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.855699    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx" (OuterVolumeSpecName: "kube-api-access-5v8kx") pod "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" (UID: "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b"). InnerVolumeSpecName "kube-api-access-5v8kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:03.856110 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.856090    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9" (OuterVolumeSpecName: "kube-api-access-b76f9") pod "97451924-94b7-41ed-960e-d02e1054172c" (UID: "97451924-94b7-41ed-960e-d02e1054172c"). InnerVolumeSpecName "kube-api-access-b76f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:03.859430 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.859407    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util" (OuterVolumeSpecName: "util") pod "97451924-94b7-41ed-960e-d02e1054172c" (UID: "97451924-94b7-41ed-960e-d02e1054172c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:03.891785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.891754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util" (OuterVolumeSpecName: "util") pod "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" (UID: "7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:03.954601 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954570    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.954601 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954597    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.954601 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954609    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-bundle\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.954833 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954620    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b76f9\" (UniqueName: \"kubernetes.io/projected/97451924-94b7-41ed-960e-d02e1054172c-kube-api-access-b76f9\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.954833 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954633    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v8kx\" (UniqueName: \"kubernetes.io/projected/7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b-kube-api-access-5v8kx\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:03.954833 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:03.954645    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97451924-94b7-41ed-960e-d02e1054172c-util\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:04.582620 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.582576    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947"
Jul 23 19:09:04.583066 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.582575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947" event={"ID":"7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b","Type":"ContainerDied","Data":"555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199"}
Jul 23 19:09:04.583066 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.582705    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="555bfaaea94b6e8282cd82ede377d4cf63d5a3bcf6eefa4b07352d6f02468199"
Jul 23 19:09:04.584433 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.584414    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg" event={"ID":"97451924-94b7-41ed-960e-d02e1054172c","Type":"ContainerDied","Data":"38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692"}
Jul 23 19:09:04.584535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.584450    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38be78b09489ab316f6b7bf0d00aacfe838b0cd74328d38c0cf4eaeaf5fd5692"
Jul 23 19:09:04.584535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:04.584424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg"
Jul 23 19:09:14.663337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663299    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"]
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663653    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663665    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663675    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663680    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663687    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663693    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663708    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663714    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663719    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663725    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663730    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663736    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663742    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663752    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663757    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="util"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663764    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663768    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663776    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663781    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663788    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663793    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="pull"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663804    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="extract"
Jul 23 19:09:14.663857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="extract"
Jul 23 19:09:14.664636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663887    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62aa6d5-b444-4f57-8208-179aa960583e" containerName="extract"
Jul 23 19:09:14.664636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663895    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="97451924-94b7-41ed-960e-d02e1054172c" containerName="extract"
Jul 23 19:09:14.664636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663900    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b" containerName="extract"
Jul 23 19:09:14.664636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.663908    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94d4809-4657-4f15-9f61-423dfaa0b77f" containerName="extract"
Jul 23 19:09:14.670619 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.670603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:14.673238 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.673216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wk67s\""
Jul 23 19:09:14.683079 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.683054    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"]
Jul 23 19:09:14.738607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.738574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdm4\" (UniqueName: \"kubernetes.io/projected/8d8ea57f-ece5-4df3-9974-c7dcf851d649-kube-api-access-2sdm4\") pod \"authorino-operator-75cc8d5cb7-q8bw5\" (UID: \"8d8ea57f-ece5-4df3-9974-c7dcf851d649\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:14.839785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.839747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdm4\" (UniqueName: \"kubernetes.io/projected/8d8ea57f-ece5-4df3-9974-c7dcf851d649-kube-api-access-2sdm4\") pod \"authorino-operator-75cc8d5cb7-q8bw5\" (UID: \"8d8ea57f-ece5-4df3-9974-c7dcf851d649\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:14.849347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.849323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdm4\" (UniqueName: \"kubernetes.io/projected/8d8ea57f-ece5-4df3-9974-c7dcf851d649-kube-api-access-2sdm4\") pod \"authorino-operator-75cc8d5cb7-q8bw5\" (UID: \"8d8ea57f-ece5-4df3-9974-c7dcf851d649\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:14.979996 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:14.979914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:15.132781 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:15.132754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"]
Jul 23 19:09:15.134166 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:15.134142    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8ea57f_ece5_4df3_9974_c7dcf851d649.slice/crio-71fb652b17037a0422f75b0a138ceb1bfb43823cd3e2a8a200bf6a3a3da021f8 WatchSource:0}: Error finding container 71fb652b17037a0422f75b0a138ceb1bfb43823cd3e2a8a200bf6a3a3da021f8: Status 404 returned error can't find the container with id 71fb652b17037a0422f75b0a138ceb1bfb43823cd3e2a8a200bf6a3a3da021f8
Jul 23 19:09:15.636322 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:15.636289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5" event={"ID":"8d8ea57f-ece5-4df3-9974-c7dcf851d649","Type":"ContainerStarted","Data":"71fb652b17037a0422f75b0a138ceb1bfb43823cd3e2a8a200bf6a3a3da021f8"}
Jul 23 19:09:17.649127 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:17.649089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5" event={"ID":"8d8ea57f-ece5-4df3-9974-c7dcf851d649","Type":"ContainerStarted","Data":"2fd55c1079d11353783b042f6a416c9235a1dcc515fd15414b2bf02338ac6305"}
Jul 23 19:09:17.649493 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:17.649159    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:17.673149 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:17.673099    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5" podStartSLOduration=1.264495803 podStartE2EDuration="3.673081959s" totalImagesPullingTime="2.408586156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:17.671139438 +0000 UTC m=+500.573725524" watchObservedRunningTime="2026-07-23 19:09:17.673081959 +0000 UTC m=+500.575668036"
Jul 23 19:09:18.070828 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.070784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"]
Jul 23 19:09:18.074154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.074135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:18.076749 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.076726    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 19:09:18.076879 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.076806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fmq6p\""
Jul 23 19:09:18.089338 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.089309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"]
Jul 23 19:09:18.170256 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.170216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk8mg\" (UniqueName: \"kubernetes.io/projected/22fcaa95-ed7f-47ba-9392-385a8c457eec-kube-api-access-fk8mg\") pod \"dns-operator-controller-manager-864dd78c4c-t28bf\" (UID: \"22fcaa95-ed7f-47ba-9392-385a8c457eec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:18.271606 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.271566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk8mg\" (UniqueName: \"kubernetes.io/projected/22fcaa95-ed7f-47ba-9392-385a8c457eec-kube-api-access-fk8mg\") pod \"dns-operator-controller-manager-864dd78c4c-t28bf\" (UID: \"22fcaa95-ed7f-47ba-9392-385a8c457eec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:18.290259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.290229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk8mg\" (UniqueName: \"kubernetes.io/projected/22fcaa95-ed7f-47ba-9392-385a8c457eec-kube-api-access-fk8mg\") pod \"dns-operator-controller-manager-864dd78c4c-t28bf\" (UID: \"22fcaa95-ed7f-47ba-9392-385a8c457eec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:18.384250 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.384156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:18.549467 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.549440    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"]
Jul 23 19:09:18.551688 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:18.551661    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fcaa95_ed7f_47ba_9392_385a8c457eec.slice/crio-461f473fb3bd20adcb9b4486ef640de6aaf4bb92ab3e76b8f50a09598c5edb1f WatchSource:0}: Error finding container 461f473fb3bd20adcb9b4486ef640de6aaf4bb92ab3e76b8f50a09598c5edb1f: Status 404 returned error can't find the container with id 461f473fb3bd20adcb9b4486ef640de6aaf4bb92ab3e76b8f50a09598c5edb1f
Jul 23 19:09:18.655768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:18.655682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf" event={"ID":"22fcaa95-ed7f-47ba-9392-385a8c457eec","Type":"ContainerStarted","Data":"461f473fb3bd20adcb9b4486ef640de6aaf4bb92ab3e76b8f50a09598c5edb1f"}
Jul 23 19:09:20.667592 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:20.667560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf" event={"ID":"22fcaa95-ed7f-47ba-9392-385a8c457eec","Type":"ContainerStarted","Data":"16d7782a7e1477eb95c90bbb62568d67a79b4faea1ca1606971669e57a74e994"}
Jul 23 19:09:20.667962 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:20.667686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:20.689687 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:20.689632    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf" podStartSLOduration=0.671601137 podStartE2EDuration="2.689614518s" totalImagesPullingTime="2.018013381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:20.688709763 +0000 UTC m=+503.591295842" watchObservedRunningTime="2026-07-23 19:09:20.689614518 +0000 UTC m=+503.592200596"
Jul 23 19:09:22.097918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.097879    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"]
Jul 23 19:09:22.101271 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.101252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.103598 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.103575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-556q5\""
Jul 23 19:09:22.115892 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.115865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"]
Jul 23 19:09:22.203694 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.203662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.203694 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.203696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjq9s\" (UniqueName: \"kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.304538 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.304501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.304692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.304544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjq9s\" (UniqueName: \"kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.304915 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.304895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.313976 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.313945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjq9s\" (UniqueName: \"kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.411230 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.411142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:22.552654 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.552626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"]
Jul 23 19:09:22.554844 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:22.554794    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf46952c_50e7_4827_8251_fd057d666943.slice/crio-e4e98810ad4e48fe272c26b49e7308b4286b318199b0bef4d569586e9e8f35d5 WatchSource:0}: Error finding container e4e98810ad4e48fe272c26b49e7308b4286b318199b0bef4d569586e9e8f35d5: Status 404 returned error can't find the container with id e4e98810ad4e48fe272c26b49e7308b4286b318199b0bef4d569586e9e8f35d5
Jul 23 19:09:22.679019 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:22.678934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv" event={"ID":"bf46952c-50e7-4827-8251-fd057d666943","Type":"ContainerStarted","Data":"e4e98810ad4e48fe272c26b49e7308b4286b318199b0bef4d569586e9e8f35d5"}
Jul 23 19:09:24.162163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.162124    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"]
Jul 23 19:09:24.165642 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.165620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:24.168188 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.168168    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-d42kn\""
Jul 23 19:09:24.179108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.179083    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"]
Jul 23 19:09:24.220704 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.220666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4cjp\" (UniqueName: \"kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp\") pod \"limitador-operator-controller-manager-859d7b78bd-64c6c\" (UID: \"be8f2241-9421-4b49-8698-c58f159632e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:24.321223 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.321187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4cjp\" (UniqueName: \"kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp\") pod \"limitador-operator-controller-manager-859d7b78bd-64c6c\" (UID: \"be8f2241-9421-4b49-8698-c58f159632e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:24.332894 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.332864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4cjp\" (UniqueName: \"kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp\") pod \"limitador-operator-controller-manager-859d7b78bd-64c6c\" (UID: \"be8f2241-9421-4b49-8698-c58f159632e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:24.475690 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.475598    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:24.614644 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.614617    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"]
Jul 23 19:09:24.616581 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:24.616544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8f2241_9421_4b49_8698_c58f159632e9.slice/crio-1dfa0814d16f35efae9f55f514b2f1cae6f7c9ddfcdaa8927532318a00f6bb5e WatchSource:0}: Error finding container 1dfa0814d16f35efae9f55f514b2f1cae6f7c9ddfcdaa8927532318a00f6bb5e: Status 404 returned error can't find the container with id 1dfa0814d16f35efae9f55f514b2f1cae6f7c9ddfcdaa8927532318a00f6bb5e
Jul 23 19:09:24.691065 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:24.691030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c" event={"ID":"be8f2241-9421-4b49-8698-c58f159632e9","Type":"ContainerStarted","Data":"1dfa0814d16f35efae9f55f514b2f1cae6f7c9ddfcdaa8927532318a00f6bb5e"}
Jul 23 19:09:26.704135 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:26.704045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c" event={"ID":"be8f2241-9421-4b49-8698-c58f159632e9","Type":"ContainerStarted","Data":"dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c"}
Jul 23 19:09:26.704527 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:26.704242    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:26.726829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:26.726750    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c" podStartSLOduration=0.935566295 podStartE2EDuration="2.726730555s" totalImagesPullingTime="1.79116426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:26.725837347 +0000 UTC m=+509.628423428" watchObservedRunningTime="2026-07-23 19:09:26.726730555 +0000 UTC m=+509.629316635"
Jul 23 19:09:28.658333 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:28.658258    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q8bw5"
Jul 23 19:09:28.717792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:28.717750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv" event={"ID":"bf46952c-50e7-4827-8251-fd057d666943","Type":"ContainerStarted","Data":"4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd"}
Jul 23 19:09:28.717965 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:28.717807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:28.738620 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:28.738568    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv" podStartSLOduration=1.033054052 podStartE2EDuration="6.738551994s" totalImagesPullingTime="5.705497942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:28.737883028 +0000 UTC m=+511.640469128" watchObservedRunningTime="2026-07-23 19:09:28.738551994 +0000 UTC m=+511.641138097"
Jul 23 19:09:30.896729 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.896692    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"]
Jul 23 19:09:30.900196 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.900176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:30.902628 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.902605    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 19:09:30.902778 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.902754    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 19:09:30.903044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.903029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zpxm7\""
Jul 23 19:09:30.904319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.904295    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"]
Jul 23 19:09:30.977320 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.977277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmhnb\" (UniqueName: \"kubernetes.io/projected/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-kube-api-access-hmhnb\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:30.977480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.977392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:30.977480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:30.977421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.078300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.078270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhnb\" (UniqueName: \"kubernetes.io/projected/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-kube-api-access-hmhnb\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.078455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.078326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.078455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.078349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.078976 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.078959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.080766 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.080743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.086014 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.085992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhnb\" (UniqueName: \"kubernetes.io/projected/9ea53e04-ba51-48e2-99de-7ee1fdb87fd4-kube-api-access-hmhnb\") pod \"kuadrant-console-plugin-66b5774985-6x2tp\" (UID: \"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.211048 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.210969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"
Jul 23 19:09:31.342294 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.342271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp"]
Jul 23 19:09:31.344382 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:31.344355    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ea53e04_ba51_48e2_99de_7ee1fdb87fd4.slice/crio-fa0f7106bc5c2ea7cfd3408119cc6009e36d406b3a3a7d57b9429812599985f7 WatchSource:0}: Error finding container fa0f7106bc5c2ea7cfd3408119cc6009e36d406b3a3a7d57b9429812599985f7: Status 404 returned error can't find the container with id fa0f7106bc5c2ea7cfd3408119cc6009e36d406b3a3a7d57b9429812599985f7
Jul 23 19:09:31.675687 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.675662    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t28bf"
Jul 23 19:09:31.734708 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:31.734675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp" event={"ID":"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4","Type":"ContainerStarted","Data":"fa0f7106bc5c2ea7cfd3408119cc6009e36d406b3a3a7d57b9429812599985f7"}
Jul 23 19:09:37.712231 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:37.712199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:39.726318 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:39.726286    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:41.450458 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.450418    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"]
Jul 23 19:09:41.451031 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.450630    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv" podUID="bf46952c-50e7-4827-8251-fd057d666943" containerName="manager" containerID="cri-o://4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd" gracePeriod=2
Jul 23 19:09:41.460524 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.460487    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"]
Jul 23 19:09:41.461192 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:41.461143    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bf46952c-50e7-4827-8251-fd057d666943\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\": pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:41.477409 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.477381    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"]
Jul 23 19:09:41.477622 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.477591    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c" podUID="be8f2241-9421-4b49-8698-c58f159632e9" containerName="manager" containerID="cri-o://dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c" gracePeriod=2
Jul 23 19:09:41.481974 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:41.480157    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:41.483497 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.483472    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:41.485768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.485390    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf46952c-50e7-4827-8251-fd057d666943" containerName="manager"
Jul 23 19:09:41.485768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.485418    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf46952c-50e7-4827-8251-fd057d666943" containerName="manager"
Jul 23 19:09:41.485768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.485738    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf46952c-50e7-4827-8251-fd057d666943" containerName="manager"
Jul 23 19:09:41.496259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.496235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.501631 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.501033    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"]
Jul 23 19:09:41.505485 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.505455    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:41.534313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.532972    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"]
Jul 23 19:09:41.534313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.533488    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8f2241-9421-4b49-8698-c58f159632e9" containerName="manager"
Jul 23 19:09:41.534313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.533508    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8f2241-9421-4b49-8698-c58f159632e9" containerName="manager"
Jul 23 19:09:41.534313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.533593    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8f2241-9421-4b49-8698-c58f159632e9" containerName="manager"
Jul 23 19:09:41.537398 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:41.537323    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:41.539095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.539061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:41.539346 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:41.539313    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:41.548875 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.548850    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"]
Jul 23 19:09:41.568912 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.568871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.568912 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.568910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv6bp\" (UniqueName: \"kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.669973 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.669934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htj57\" (UniqueName: \"kubernetes.io/projected/30c219e9-d394-4f87-b2a5-2e8cb5ffd79c-kube-api-access-htj57\") pod \"limitador-operator-controller-manager-859d7b78bd-xnb74\" (UID: \"30c219e9-d394-4f87-b2a5-2e8cb5ffd79c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:41.670142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.670055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.670142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.670093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv6bp\" (UniqueName: \"kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.670500 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.670477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.681332 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.681307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv6bp\" (UniqueName: \"kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qslpq\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.771300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.771266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htj57\" (UniqueName: \"kubernetes.io/projected/30c219e9-d394-4f87-b2a5-2e8cb5ffd79c-kube-api-access-htj57\") pod \"limitador-operator-controller-manager-859d7b78bd-xnb74\" (UID: \"30c219e9-d394-4f87-b2a5-2e8cb5ffd79c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:41.794931 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.794899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htj57\" (UniqueName: \"kubernetes.io/projected/30c219e9-d394-4f87-b2a5-2e8cb5ffd79c-kube-api-access-htj57\") pod \"limitador-operator-controller-manager-859d7b78bd-xnb74\" (UID: \"30c219e9-d394-4f87-b2a5-2e8cb5ffd79c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:41.877072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.877034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:41.884346 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:41.884320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:54.474520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.474474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"]
Jul 23 19:09:54.477703 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:54.477667    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c219e9_d394_4f87_b2a5_2e8cb5ffd79c.slice/crio-ab561b665b5025def93e5eb073b427007430b28033d05ef98e7852c0d171a69c WatchSource:0}: Error finding container ab561b665b5025def93e5eb073b427007430b28033d05ef98e7852c0d171a69c: Status 404 returned error can't find the container with id ab561b665b5025def93e5eb073b427007430b28033d05ef98e7852c0d171a69c
Jul 23 19:09:54.500566 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.500541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:54.688019 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:54.687986    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262d8a8c_088d_4cb9_808f_fae76d86be72.slice/crio-a1feb9771c98ca5e86d14165a1e2311d797adcef52188dc9a12d9d7f9e27e334 WatchSource:0}: Error finding container a1feb9771c98ca5e86d14165a1e2311d797adcef52188dc9a12d9d7f9e27e334: Status 404 returned error can't find the container with id a1feb9771c98ca5e86d14165a1e2311d797adcef52188dc9a12d9d7f9e27e334
Jul 23 19:09:54.731878 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.731853    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.734067 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.734036    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.756879 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.756855    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:54.759281 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.759245    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.761067 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.761033    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:54.792225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.792204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4cjp\" (UniqueName: \"kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp\") pod \"be8f2241-9421-4b49-8698-c58f159632e9\" (UID: \"be8f2241-9421-4b49-8698-c58f159632e9\") "
Jul 23 19:09:54.794375 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.794351    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp" (OuterVolumeSpecName: "kube-api-access-h4cjp") pod "be8f2241-9421-4b49-8698-c58f159632e9" (UID: "be8f2241-9421-4b49-8698-c58f159632e9"). InnerVolumeSpecName "kube-api-access-h4cjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:54.869969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.869931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp" event={"ID":"9ea53e04-ba51-48e2-99de-7ee1fdb87fd4","Type":"ContainerStarted","Data":"32247d76ad42c1effd9a5edfa19b539fec52440a22dbaf1c8ba114bb03c93a3d"}
Jul 23 19:09:54.871652 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.871623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74" event={"ID":"30c219e9-d394-4f87-b2a5-2e8cb5ffd79c","Type":"ContainerStarted","Data":"56eda70a09f6534748aa396e7517b406002ea12dcb8e7b765a0a60ee324ca3b5"}
Jul 23 19:09:54.871794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.871701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74" event={"ID":"30c219e9-d394-4f87-b2a5-2e8cb5ffd79c","Type":"ContainerStarted","Data":"ab561b665b5025def93e5eb073b427007430b28033d05ef98e7852c0d171a69c"}
Jul 23 19:09:54.871794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.871722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:09:54.871933 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.871894    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:54.873095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.873071    2563 generic.go:356] "Generic (PLEG): container finished" podID="bf46952c-50e7-4827-8251-fd057d666943" containerID="4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd" exitCode=0
Jul 23 19:09:54.873200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.873123    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:54.873200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.873146    2563 scope.go:117] "RemoveContainer" containerID="4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd"
Jul 23 19:09:54.874631 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.874612    2563 generic.go:356] "Generic (PLEG): container finished" podID="be8f2241-9421-4b49-8698-c58f159632e9" containerID="dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c" exitCode=0
Jul 23 19:09:54.874718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.874645    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.876715 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.876681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" event={"ID":"262d8a8c-088d-4cb9-808f-fae76d86be72","Type":"ContainerStarted","Data":"51d7160baca7e8ab6c317368aa9c1f958b666ae9344013661338f6c4cb9de1ab"}
Jul 23 19:09:54.876837 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.876733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" event={"ID":"262d8a8c-088d-4cb9-808f-fae76d86be72","Type":"ContainerStarted","Data":"a1feb9771c98ca5e86d14165a1e2311d797adcef52188dc9a12d9d7f9e27e334"}
Jul 23 19:09:54.876837 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.876765    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:54.882084 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.882066    2563 scope.go:117] "RemoveContainer" containerID="4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd"
Jul 23 19:09:54.882389 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.882368    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd\": container with ID starting with 4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd not found: ID does not exist" containerID="4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd"
Jul 23 19:09:54.882462 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.882398    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd"} err="failed to get container status \"4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd\": rpc error: code = NotFound desc = could not find container \"4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd\": container with ID starting with 4e9e45284038bbaec28e9797180013309986b1aac6c73b2664e45d283a27fdfd not found: ID does not exist"
Jul 23 19:09:54.882462 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.882415    2563 scope.go:117] "RemoveContainer" containerID="dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c"
Jul 23 19:09:54.889397 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.889371    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.890460 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.890413    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6x2tp" podStartSLOduration=1.475032985 podStartE2EDuration="24.890396851s" totalImagesPullingTime="23.415363866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:54.887546846 +0000 UTC m=+537.790132938" watchObservedRunningTime="2026-07-23 19:09:54.890396851 +0000 UTC m=+537.792982929"
Jul 23 19:09:54.891034 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.891010    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:54.891626 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.891611    2563 scope.go:117] "RemoveContainer" containerID="dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c"
Jul 23 19:09:54.891909 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.891889    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c\": container with ID starting with dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c not found: ID does not exist" containerID="dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c"
Jul 23 19:09:54.891996 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.891914    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c"} err="failed to get container status \"dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c\": rpc error: code = NotFound desc = could not find container \"dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c\": container with ID starting with dac7eaa120bc712f9e54ad6a1c54e25140ce39b9f72da4bd1797b330cdfcca0c not found: ID does not exist"
Jul 23 19:09:54.893119 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.893100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjq9s\" (UniqueName: \"kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s\") pod \"bf46952c-50e7-4827-8251-fd057d666943\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") "
Jul 23 19:09:54.893197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.893139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume\") pod \"bf46952c-50e7-4827-8251-fd057d666943\" (UID: \"bf46952c-50e7-4827-8251-fd057d666943\") "
Jul 23 19:09:54.893364 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.893350    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4cjp\" (UniqueName: \"kubernetes.io/projected/be8f2241-9421-4b49-8698-c58f159632e9-kube-api-access-h4cjp\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:54.893577 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.893559    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf46952c-50e7-4827-8251-fd057d666943" (UID: "bf46952c-50e7-4827-8251-fd057d666943"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:54.895247 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.895226    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s" (OuterVolumeSpecName: "kube-api-access-cjq9s") pod "bf46952c-50e7-4827-8251-fd057d666943" (UID: "bf46952c-50e7-4827-8251-fd057d666943"). InnerVolumeSpecName "kube-api-access-cjq9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:54.911076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.911018    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" podStartSLOduration=13.910998836 podStartE2EDuration="13.910998836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:54.909794969 +0000 UTC m=+537.812381047" watchObservedRunningTime="2026-07-23 19:09:54.910998836 +0000 UTC m=+537.813584918"
Jul 23 19:09:54.928456 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.928374    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74" podStartSLOduration=13.928351963 podStartE2EDuration="13.928351963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:54.927666898 +0000 UTC m=+537.830252976" watchObservedRunningTime="2026-07-23 19:09:54.928351963 +0000 UTC m=+537.830938042"
Jul 23 19:09:54.929495 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:54.929468    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:54.994665 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.994579    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjq9s\" (UniqueName: \"kubernetes.io/projected/bf46952c-50e7-4827-8251-fd057d666943-kube-api-access-cjq9s\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:54.994665 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:54.994611    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf46952c-50e7-4827-8251-fd057d666943-extensions-socket-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:55.183483 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:55.183448    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wwqbv\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="bf46952c-50e7-4827-8251-fd057d666943" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wwqbv"
Jul 23 19:09:55.185410 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:09:55.185388    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-64c6c\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="be8f2241-9421-4b49-8698-c58f159632e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-64c6c"
Jul 23 19:09:55.522792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:55.522762    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8f2241-9421-4b49-8698-c58f159632e9" path="/var/lib/kubelet/pods/be8f2241-9421-4b49-8698-c58f159632e9/volumes"
Jul 23 19:09:55.523166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:55.523118    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf46952c-50e7-4827-8251-fd057d666943" path="/var/lib/kubelet/pods/bf46952c-50e7-4827-8251-fd057d666943/volumes"
Jul 23 19:09:57.490145 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.490112    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:57.490568 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.490297    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" podUID="262d8a8c-088d-4cb9-808f-fae76d86be72" containerName="manager" containerID="cri-o://51d7160baca7e8ab6c317368aa9c1f958b666ae9344013661338f6c4cb9de1ab" gracePeriod=10
Jul 23 19:09:57.737659 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.737575    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:09:57.768566 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.768535    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:09:57.768717 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.768628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:57.895854 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.895801    2563 generic.go:356] "Generic (PLEG): container finished" podID="262d8a8c-088d-4cb9-808f-fae76d86be72" containerID="51d7160baca7e8ab6c317368aa9c1f958b666ae9344013661338f6c4cb9de1ab" exitCode=0
Jul 23 19:09:57.896028 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.895877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" event={"ID":"262d8a8c-088d-4cb9-808f-fae76d86be72","Type":"ContainerDied","Data":"51d7160baca7e8ab6c317368aa9c1f958b666ae9344013661338f6c4cb9de1ab"}
Jul 23 19:09:57.919047 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.919015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdxjj\" (UniqueName: \"kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:57.919126 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:57.919073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.019999 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.019957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdxjj\" (UniqueName: \"kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.020156 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.020031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.020435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.020416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.029431 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.029402    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdxjj\" (UniqueName: \"kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj\") pod \"kuadrant-operator-controller-manager-86878c769f-t6qdt\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.078037 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.078003    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.303823 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.303792    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:58.423922 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.423886    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume\") pod \"262d8a8c-088d-4cb9-808f-fae76d86be72\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") "
Jul 23 19:09:58.424093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.423939    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv6bp\" (UniqueName: \"kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp\") pod \"262d8a8c-088d-4cb9-808f-fae76d86be72\" (UID: \"262d8a8c-088d-4cb9-808f-fae76d86be72\") "
Jul 23 19:09:58.424359 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.424332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "262d8a8c-088d-4cb9-808f-fae76d86be72" (UID: "262d8a8c-088d-4cb9-808f-fae76d86be72"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:09:58.426251 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.426226    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp" (OuterVolumeSpecName: "kube-api-access-qv6bp") pod "262d8a8c-088d-4cb9-808f-fae76d86be72" (UID: "262d8a8c-088d-4cb9-808f-fae76d86be72"). InnerVolumeSpecName "kube-api-access-qv6bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:09:58.452345 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.452309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:09:58.455450 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:09:58.455421    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7835e023_56a1_4d04_bf13_b52d3bd5b6c6.slice/crio-0ad3852f5a75768c162af9c0053c46dcc3f13e001b49fd82e22a606d501098e9 WatchSource:0}: Error finding container 0ad3852f5a75768c162af9c0053c46dcc3f13e001b49fd82e22a606d501098e9: Status 404 returned error can't find the container with id 0ad3852f5a75768c162af9c0053c46dcc3f13e001b49fd82e22a606d501098e9
Jul 23 19:09:58.524773 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.524748    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/262d8a8c-088d-4cb9-808f-fae76d86be72-extensions-socket-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:58.525062 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.524775    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv6bp\" (UniqueName: \"kubernetes.io/projected/262d8a8c-088d-4cb9-808f-fae76d86be72-kube-api-access-qv6bp\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:09:58.902438 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.902354    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"
Jul 23 19:09:58.902438 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.902391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq" event={"ID":"262d8a8c-088d-4cb9-808f-fae76d86be72","Type":"ContainerDied","Data":"a1feb9771c98ca5e86d14165a1e2311d797adcef52188dc9a12d9d7f9e27e334"}
Jul 23 19:09:58.902617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.902455    2563 scope.go:117] "RemoveContainer" containerID="51d7160baca7e8ab6c317368aa9c1f958b666ae9344013661338f6c4cb9de1ab"
Jul 23 19:09:58.904279 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.904252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" event={"ID":"7835e023-56a1-4d04-bf13-b52d3bd5b6c6","Type":"ContainerStarted","Data":"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"}
Jul 23 19:09:58.904390 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.904299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" event={"ID":"7835e023-56a1-4d04-bf13-b52d3bd5b6c6","Type":"ContainerStarted","Data":"0ad3852f5a75768c162af9c0053c46dcc3f13e001b49fd82e22a606d501098e9"}
Jul 23 19:09:58.904390 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.904328    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:09:58.928552 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.928344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" podStartSLOduration=1.928328899 podStartE2EDuration="1.928328899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:09:58.927304757 +0000 UTC m=+541.829890838" watchObservedRunningTime="2026-07-23 19:09:58.928328899 +0000 UTC m=+541.830914977"
Jul 23 19:09:58.943680 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.943647    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:58.949152 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:58.949131    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qslpq"]
Jul 23 19:09:59.522938 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:09:59.522905    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="262d8a8c-088d-4cb9-808f-fae76d86be72" path="/var/lib/kubelet/pods/262d8a8c-088d-4cb9-808f-fae76d86be72/volumes"
Jul 23 19:10:05.886216 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:05.886186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xnb74"
Jul 23 19:10:09.912898 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:09.912869    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:10:32.646138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.646103    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:10:32.646610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.646496    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262d8a8c-088d-4cb9-808f-fae76d86be72" containerName="manager"
Jul 23 19:10:32.646610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.646508    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="262d8a8c-088d-4cb9-808f-fae76d86be72" containerName="manager"
Jul 23 19:10:32.646610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.646581    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="262d8a8c-088d-4cb9-808f-fae76d86be72" containerName="manager"
Jul 23 19:10:32.712376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.712338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:10:32.712376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.712380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:10:32.712565 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.712452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.715286 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.715262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 19:10:32.794416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.794377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e0c60f38-b099-4d14-be18-0ab0a9404e3c-config-file\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.794603 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.794430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4g5r\" (UniqueName: \"kubernetes.io/projected/e0c60f38-b099-4d14-be18-0ab0a9404e3c-kube-api-access-k4g5r\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.894947 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.894906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e0c60f38-b099-4d14-be18-0ab0a9404e3c-config-file\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.894947 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.894956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4g5r\" (UniqueName: \"kubernetes.io/projected/e0c60f38-b099-4d14-be18-0ab0a9404e3c-kube-api-access-k4g5r\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.895605 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.895571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e0c60f38-b099-4d14-be18-0ab0a9404e3c-config-file\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:32.906213 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:32.906138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4g5r\" (UniqueName: \"kubernetes.io/projected/e0c60f38-b099-4d14-be18-0ab0a9404e3c-kube-api-access-k4g5r\") pod \"limitador-limitador-54d6d988d5-5xfcx\" (UID: \"e0c60f38-b099-4d14-be18-0ab0a9404e3c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:33.023137 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:33.023099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:33.175118 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:33.175086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:10:33.176592 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:10:33.176564    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c60f38_b099_4d14_be18_0ab0a9404e3c.slice/crio-a104122130e5624adb11764e3be31f4d297b8632853388cf94bffee98117d194 WatchSource:0}: Error finding container a104122130e5624adb11764e3be31f4d297b8632853388cf94bffee98117d194: Status 404 returned error can't find the container with id a104122130e5624adb11764e3be31f4d297b8632853388cf94bffee98117d194
Jul 23 19:10:34.089590 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:34.089546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx" event={"ID":"e0c60f38-b099-4d14-be18-0ab0a9404e3c","Type":"ContainerStarted","Data":"a104122130e5624adb11764e3be31f4d297b8632853388cf94bffee98117d194"}
Jul 23 19:10:36.102520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:36.102480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx" event={"ID":"e0c60f38-b099-4d14-be18-0ab0a9404e3c","Type":"ContainerStarted","Data":"c1a2ec24e24192ded34e60cfa70cf4b7b8006c9c70dd01c7159a145c60d69e8b"}
Jul 23 19:10:36.102926 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:36.102544    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:10:36.123296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:36.123230    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx" podStartSLOduration=1.68163603 podStartE2EDuration="4.123210263s" totalImagesPullingTime="2.441574233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:10:36.12069829 +0000 UTC m=+579.023284390" watchObservedRunningTime="2026-07-23 19:10:36.123210263 +0000 UTC m=+579.025796373"
Jul 23 19:10:47.109959 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:10:47.109929    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"
Jul 23 19:11:51.212463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:11:51.212429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:12:19.256159 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:19.256121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:12:30.904703 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.904667    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"]
Jul 23 19:12:30.908341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.908314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:30.911745 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.911715    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 19:12:30.911917 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.911782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g5frh\""
Jul 23 19:12:30.912044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.912020    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 19:12:30.912179 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.912099    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 19:12:30.922212 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:30.922186    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"]
Jul 23 19:12:31.052298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tdh8\" (UniqueName: \"kubernetes.io/projected/b2581274-426e-4405-ba6d-da9973234bea-kube-api-access-8tdh8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.052475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.052475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.052475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2581274-426e-4405-ba6d-da9973234bea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.052590 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.052590 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.052498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.153908 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.153868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.153943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2581274-426e-4405-ba6d-da9973234bea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.153979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.153997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.154023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tdh8\" (UniqueName: \"kubernetes.io/projected/b2581274-426e-4405-ba6d-da9973234bea-kube-api-access-8tdh8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.154048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154399 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.154349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154399 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.154392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.154469 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.154420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.156248 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.156189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2581274-426e-4405-ba6d-da9973234bea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.156468 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.156450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2581274-426e-4405-ba6d-da9973234bea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.163660 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.163635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tdh8\" (UniqueName: \"kubernetes.io/projected/b2581274-426e-4405-ba6d-da9973234bea-kube-api-access-8tdh8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vh2kx\" (UID: \"b2581274-426e-4405-ba6d-da9973234bea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.221366 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.221329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:31.363136 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.363106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"]
Jul 23 19:12:31.364079 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:12:31.364054    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2581274_426e_4405_ba6d_da9973234bea.slice/crio-936a0fc8e28f8c364e0da10eb65ed0ab1cb0a51d799335dbbaf23fb8ae9ab8df WatchSource:0}: Error finding container 936a0fc8e28f8c364e0da10eb65ed0ab1cb0a51d799335dbbaf23fb8ae9ab8df: Status 404 returned error can't find the container with id 936a0fc8e28f8c364e0da10eb65ed0ab1cb0a51d799335dbbaf23fb8ae9ab8df
Jul 23 19:12:31.365410 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.365391    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:12:31.695669 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.695630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx" event={"ID":"b2581274-426e-4405-ba6d-da9973234bea","Type":"ContainerStarted","Data":"936a0fc8e28f8c364e0da10eb65ed0ab1cb0a51d799335dbbaf23fb8ae9ab8df"}
Jul 23 19:12:31.875547 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:31.875514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:12:32.702052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:32.701970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx" event={"ID":"b2581274-426e-4405-ba6d-da9973234bea","Type":"ContainerStarted","Data":"dd87f983ac52174948e917145e46fb3fe2eafa627039cae9697d835d210f7c67"}
Jul 23 19:12:32.723834 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:32.723755    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx" podStartSLOduration=1.6843675569999998 podStartE2EDuration="2.723735872s" totalImagesPullingTime="1.039368315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:12:32.72309653 +0000 UTC m=+695.625682632" watchObservedRunningTime="2026-07-23 19:12:32.723735872 +0000 UTC m=+695.626321951"
Jul 23 19:12:38.885395 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:38.885356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:12:41.221927 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:41.221896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:41.222385 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:41.222013    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:41.234522 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:41.234491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:41.761914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:41.761887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vh2kx"
Jul 23 19:12:58.174741 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:12:58.174646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:13:04.368464 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.368426    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"]
Jul 23 19:13:04.372607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.372587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.374774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.374755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 19:13:04.384323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.384298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"]
Jul 23 19:13:04.435869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.435827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.436036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.435896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b23f24-94ee-4006-b191-13e56585b64a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.436036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.435946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.436036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.435975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7qn\" (UniqueName: \"kubernetes.io/projected/49b23f24-94ee-4006-b191-13e56585b64a-kube-api-access-7r7qn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.436143 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.436067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.436143 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.436118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537284 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b23f24-94ee-4006-b191-13e56585b64a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7qn\" (UniqueName: \"kubernetes.io/projected/49b23f24-94ee-4006-b191-13e56585b64a-kube-api-access-7r7qn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537912 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.537979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.537908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.539757 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.539729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49b23f24-94ee-4006-b191-13e56585b64a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.539988 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.539970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49b23f24-94ee-4006-b191-13e56585b64a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.545548 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.545528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7qn\" (UniqueName: \"kubernetes.io/projected/49b23f24-94ee-4006-b191-13e56585b64a-kube-api-access-7r7qn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v9q7w\" (UID: \"49b23f24-94ee-4006-b191-13e56585b64a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.684856 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.684745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:04.824541 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.824476    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"]
Jul 23 19:13:04.826920 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:13:04.826892    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b23f24_94ee_4006_b191_13e56585b64a.slice/crio-02578ca3cce2f8da0eca81f50659593178c8ca163c648b9926b2093d6b3454ce WatchSource:0}: Error finding container 02578ca3cce2f8da0eca81f50659593178c8ca163c648b9926b2093d6b3454ce: Status 404 returned error can't find the container with id 02578ca3cce2f8da0eca81f50659593178c8ca163c648b9926b2093d6b3454ce
Jul 23 19:13:04.873452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:04.873417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w" event={"ID":"49b23f24-94ee-4006-b191-13e56585b64a","Type":"ContainerStarted","Data":"02578ca3cce2f8da0eca81f50659593178c8ca163c648b9926b2093d6b3454ce"}
Jul 23 19:13:05.880423 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:05.880379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w" event={"ID":"49b23f24-94ee-4006-b191-13e56585b64a","Type":"ContainerStarted","Data":"ef752b5b093054a146feb9f34953e3ead1cafdf0d96c64366868c2f5d6bf4b3b"}
Jul 23 19:13:05.903114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:05.903065    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w" podStartSLOduration=1.59986765 podStartE2EDuration="1.903048967s" totalImagesPullingTime="303.181317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:13:05.901598019 +0000 UTC m=+728.804184118" watchObservedRunningTime="2026-07-23 19:13:05.903048967 +0000 UTC m=+728.805635073"
Jul 23 19:13:07.581794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:07.581757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:13:14.685594 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:14.685556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:14.686039 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:14.685610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:14.697961 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:14.697937    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:14.940763 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:14.940676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v9q7w"
Jul 23 19:13:18.477098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.477063    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"]
Jul 23 19:13:18.480616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.480592    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.483234 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.483211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 19:13:18.494796 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.494769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"]
Jul 23 19:13:18.562543 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.562543 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.562787 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f477e24-d34e-41ab-900b-84984d94ded4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.562787 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjp7\" (UniqueName: \"kubernetes.io/projected/0f477e24-d34e-41ab-900b-84984d94ded4-kube-api-access-7sjp7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.562787 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.562957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.562794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664230 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664230 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664445 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f477e24-d34e-41ab-900b-84984d94ded4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664445 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjp7\" (UniqueName: \"kubernetes.io/projected/0f477e24-d34e-41ab-900b-84984d94ded4-kube-api-access-7sjp7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664445 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664588 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664674 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664674 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.664908 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.664885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.666657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.666637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f477e24-d34e-41ab-900b-84984d94ded4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.666914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.666895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f477e24-d34e-41ab-900b-84984d94ded4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.673363 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.673344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjp7\" (UniqueName: \"kubernetes.io/projected/0f477e24-d34e-41ab-900b-84984d94ded4-kube-api-access-7sjp7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9\" (UID: \"0f477e24-d34e-41ab-900b-84984d94ded4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.790315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.790275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:18.933918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.933887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"]
Jul 23 19:13:18.937340 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:13:18.937307    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f477e24_d34e_41ab_900b_84984d94ded4.slice/crio-4de7ddf3e5c2bcdda2bf585822487fca730f9e0b761914c651cf08efb01dba48 WatchSource:0}: Error finding container 4de7ddf3e5c2bcdda2bf585822487fca730f9e0b761914c651cf08efb01dba48: Status 404 returned error can't find the container with id 4de7ddf3e5c2bcdda2bf585822487fca730f9e0b761914c651cf08efb01dba48
Jul 23 19:13:18.950956 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:18.950921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9" event={"ID":"0f477e24-d34e-41ab-900b-84984d94ded4","Type":"ContainerStarted","Data":"4de7ddf3e5c2bcdda2bf585822487fca730f9e0b761914c651cf08efb01dba48"}
Jul 23 19:13:19.957665 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:19.957631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9" event={"ID":"0f477e24-d34e-41ab-900b-84984d94ded4","Type":"ContainerStarted","Data":"c51d2def2e9c30bc63eea04635df921967053c92b2436fbe0bdc1feca32bd5a5"}
Jul 23 19:13:19.977484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:19.977439    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9" podStartSLOduration=1.784799913 podStartE2EDuration="1.977423201s" totalImagesPullingTime="192.623288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:13:19.97662973 +0000 UTC m=+742.879215809" watchObservedRunningTime="2026-07-23 19:13:19.977423201 +0000 UTC m=+742.880009279"
Jul 23 19:13:20.903209 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:20.903171    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:13:28.790841 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:28.790769    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:28.790841 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:28.790833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:28.803998 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:28.803967    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:13:29.019166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:13:29.019134    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9"
Jul 23 19:14:46.681424 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:14:46.681387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:14:56.075913 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:14:56.075869    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:15:06.974184 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:15:06.974147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:16:49.376726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:16:49.376690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:16:57.691843 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:16:57.691796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:17:29.069149 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:17:29.069110    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:17:38.493921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:17:38.493883    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:17:47.469187 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:17:47.469150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:17:59.178348 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:17:59.178312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:18:10.395062 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:18:10.395029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:18:22.278517 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:18:22.278483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:18:32.792764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:18:32.792722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:18:42.573259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:18:42.573223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:19:04.880692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:19:04.880602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:19:17.084541 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:19:17.084505    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:19:42.677798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:19:42.677764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:19:52.272801 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:19:52.272766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:20:06.783880 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:20:06.783841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:20:17.469800 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:20:17.469762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:20:26.681148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:20:26.681069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:20:35.975430 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:20:35.975394    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:20:45.372227 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:20:45.372194    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:21:11.414918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:11.414876    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:21:11.415527 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:11.415122    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" podUID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" containerName="manager" containerID="cri-o://d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5" gracePeriod=10
Jul 23 19:21:14.263133 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.263110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:21:14.277639 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.277617    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume\") pod \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") "
Jul 23 19:21:14.277762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.277664    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdxjj\" (UniqueName: \"kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj\") pod \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\" (UID: \"7835e023-56a1-4d04-bf13-b52d3bd5b6c6\") "
Jul 23 19:21:14.277965 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.277942    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7835e023-56a1-4d04-bf13-b52d3bd5b6c6" (UID: "7835e023-56a1-4d04-bf13-b52d3bd5b6c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:21:14.279648 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.279623    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj" (OuterVolumeSpecName: "kube-api-access-bdxjj") pod "7835e023-56a1-4d04-bf13-b52d3bd5b6c6" (UID: "7835e023-56a1-4d04-bf13-b52d3bd5b6c6"). InnerVolumeSpecName "kube-api-access-bdxjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:21:14.374068 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.373979    2563 generic.go:356] "Generic (PLEG): container finished" podID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" containerID="d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5" exitCode=0
Jul 23 19:21:14.374068 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.374033    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"
Jul 23 19:21:14.374255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.374072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" event={"ID":"7835e023-56a1-4d04-bf13-b52d3bd5b6c6","Type":"ContainerDied","Data":"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"}
Jul 23 19:21:14.374255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.374167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt" event={"ID":"7835e023-56a1-4d04-bf13-b52d3bd5b6c6","Type":"ContainerDied","Data":"0ad3852f5a75768c162af9c0053c46dcc3f13e001b49fd82e22a606d501098e9"}
Jul 23 19:21:14.374255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.374201    2563 scope.go:117] "RemoveContainer" containerID="d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"
Jul 23 19:21:14.378196 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.378171    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-extensions-socket-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:21:14.378196 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.378193    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdxjj\" (UniqueName: \"kubernetes.io/projected/7835e023-56a1-4d04-bf13-b52d3bd5b6c6-kube-api-access-bdxjj\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:21:14.384776 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.384758    2563 scope.go:117] "RemoveContainer" containerID="d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"
Jul 23 19:21:14.385053 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:21:14.385033    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5\": container with ID starting with d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5 not found: ID does not exist" containerID="d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"
Jul 23 19:21:14.385171 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.385058    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5"} err="failed to get container status \"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5\": rpc error: code = NotFound desc = could not find container \"d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5\": container with ID starting with d492730c576d04b42589e0b9525551b84c552d848f517ce654ab891e9049d3c5 not found: ID does not exist"
Jul 23 19:21:14.397899 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.397874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:21:14.400757 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:14.400735    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t6qdt"]
Jul 23 19:21:15.521906 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:21:15.521870    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" path="/var/lib/kubelet/pods/7835e023-56a1-4d04-bf13-b52d3bd5b6c6/volumes"
Jul 23 19:22:36.530461 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:22:36.530429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:22:47.529988 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:22:47.529958    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:22:56.424442 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:22:56.424410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:08.631715 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:08.631673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:20.026668 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:20.026581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:26.426803 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:26.426767    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:34.723202 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:34.723162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:46.133256 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:46.133222    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:23:55.227881 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:23:55.227847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:06.227149 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:06.227117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:17.430741 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:17.430712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:24.925746 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:24.925712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:45.727257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:45.727217    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:53.230061 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.229975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:24:53.986724 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.986689    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:24:53.987252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.987237    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" containerName="manager"
Jul 23 19:24:53.987300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.987257    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" containerName="manager"
Jul 23 19:24:53.987349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.987338    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7835e023-56a1-4d04-bf13-b52d3bd5b6c6" containerName="manager"
Jul 23 19:24:53.990688 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.990672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:53.993949 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:53.993927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-055d1f1e-kserve-self-signed-certs\""
Jul 23 19:24:54.014403 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.014381    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:24:54.152835 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.152784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.153005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.152846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzd82\" (UniqueName: \"kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.153005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.152950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.153081 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.153016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.153081 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.153060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.153144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.153084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.253908 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.253875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.253921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.253940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.253978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.253995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzd82\" (UniqueName: \"kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.254022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.254350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.254360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.254570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.254393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.256416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.256384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.256597 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.256580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.261476 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.261452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzd82\" (UniqueName: \"kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82\") pod \"e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") " pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.299992 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.299962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:24:54.439306 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.439275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:24:54.441078 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:24:54.441051    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e8ece50_e577_45c6_8c48_d4d84d0ff902.slice/crio-e6c9962bfbc4f51c9af78b4f80d1c27eb553dfef25099d57efe33fb0efa17cc0 WatchSource:0}: Error finding container e6c9962bfbc4f51c9af78b4f80d1c27eb553dfef25099d57efe33fb0efa17cc0: Status 404 returned error can't find the container with id e6c9962bfbc4f51c9af78b4f80d1c27eb553dfef25099d57efe33fb0efa17cc0
Jul 23 19:24:54.464997 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:54.464957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" event={"ID":"3e8ece50-e577-45c6-8c48-d4d84d0ff902","Type":"ContainerStarted","Data":"e6c9962bfbc4f51c9af78b4f80d1c27eb553dfef25099d57efe33fb0efa17cc0"}
Jul 23 19:24:55.471828 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:24:55.471777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" event={"ID":"3e8ece50-e577-45c6-8c48-d4d84d0ff902","Type":"ContainerStarted","Data":"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"}
Jul 23 19:25:04.300987 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:04.300956    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:04.301364 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:04.300998    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:04.312942 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:04.312918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:04.330903 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:04.330865    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" podStartSLOduration=11.330852137 podStartE2EDuration="11.330852137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:24:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:24:55.493849976 +0000 UTC m=+1438.396436125" watchObservedRunningTime="2026-07-23 19:25:04.330852137 +0000 UTC m=+1447.233438231"
Jul 23 19:25:04.530003 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:04.529974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:05.026345 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:05.026315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:25:05.962564 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:05.962529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:05.966300 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:05.966278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:05.968665 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:05.968648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-055d1f1e-kserve-self-signed-certs\""
Jul 23 19:25:05.977174 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:05.977153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:06.159648 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.159808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.159808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzfv7\" (UniqueName: \"kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.159808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.159808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.159808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.159801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261041 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzfv7\" (UniqueName: \"kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261473 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261538 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.261593 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.261578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.263298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.263277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.263541 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.263525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.268955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.268932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzfv7\" (UniqueName: \"kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7\") pod \"e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") " pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.278672 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.278654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:06.412330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.412302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:06.414000 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:25:06.413970    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd54d082c_c636_40ba_bf81_ac99d3a5e9e7.slice/crio-17166e1b1ca479e3a3aab684815695ff0ca1cf664a2ba3d851ffbf7e7963aafc WatchSource:0}: Error finding container 17166e1b1ca479e3a3aab684815695ff0ca1cf664a2ba3d851ffbf7e7963aafc: Status 404 returned error can't find the container with id 17166e1b1ca479e3a3aab684815695ff0ca1cf664a2ba3d851ffbf7e7963aafc
Jul 23 19:25:06.536387 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.536307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" event={"ID":"d54d082c-c636-40ba-bf81-ac99d3a5e9e7","Type":"ContainerStarted","Data":"7926a2c791209c8fd4a48761cd64b58e8a62ff8b16cda1959d5d219985c35c25"}
Jul 23 19:25:06.536387 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.536350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" event={"ID":"d54d082c-c636-40ba-bf81-ac99d3a5e9e7","Type":"ContainerStarted","Data":"17166e1b1ca479e3a3aab684815695ff0ca1cf664a2ba3d851ffbf7e7963aafc"}
Jul 23 19:25:06.552316 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:06.552271    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" podStartSLOduration=1.552256384 podStartE2EDuration="1.552256384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:25:06.551653119 +0000 UTC m=+1449.454239223" watchObservedRunningTime="2026-07-23 19:25:06.552256384 +0000 UTC m=+1449.454842462"
Jul 23 19:25:15.337783 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:15.337748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:25:16.279378 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:16.279345    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:16.279569 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:16.279394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:16.291677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:16.291655    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:16.601740 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:16.601663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:22.929024 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:22.928994    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:25:24.461981 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:24.461946    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:25.930849 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:25.930806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:25:37.949052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:37.949000    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" podUID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" containerName="main" containerID="cri-o://f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc" gracePeriod=45
Jul 23 19:25:38.208625 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.208572    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:38.320048 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320017    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320048 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320083    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzd82\" (UniqueName: \"kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320255 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache\") pod \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\" (UID: \"3e8ece50-e577-45c6-8c48-d4d84d0ff902\") "
Jul 23 19:25:38.320438 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320300    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:38.320499 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home" (OuterVolumeSpecName: "home") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:38.320581 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320549    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:38.320581 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320563    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.320751 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.320600    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.322249 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.322226    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm" (OuterVolumeSpecName: "dshm") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:38.322356 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.322307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:25:38.322356 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.322338    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82" (OuterVolumeSpecName: "kube-api-access-pzd82") pod "3e8ece50-e577-45c6-8c48-d4d84d0ff902" (UID: "3e8ece50-e577-45c6-8c48-d4d84d0ff902"). InnerVolumeSpecName "kube-api-access-pzd82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:25:38.421636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.421601    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8ece50-e577-45c6-8c48-d4d84d0ff902-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.421636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.421631    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.421636 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.421639    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8ece50-e577-45c6-8c48-d4d84d0ff902-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.421889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.421648    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzd82\" (UniqueName: \"kubernetes.io/projected/3e8ece50-e577-45c6-8c48-d4d84d0ff902-kube-api-access-pzd82\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:38.706047 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.706016    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" containerID="f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc" exitCode=0
Jul 23 19:25:38.706241 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.706056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"
Jul 23 19:25:38.706241 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.706100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" event={"ID":"3e8ece50-e577-45c6-8c48-d4d84d0ff902","Type":"ContainerDied","Data":"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"}
Jul 23 19:25:38.706241 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.706144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj" event={"ID":"3e8ece50-e577-45c6-8c48-d4d84d0ff902","Type":"ContainerDied","Data":"e6c9962bfbc4f51c9af78b4f80d1c27eb553dfef25099d57efe33fb0efa17cc0"}
Jul 23 19:25:38.706241 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.706165    2563 scope.go:117] "RemoveContainer" containerID="f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"
Jul 23 19:25:38.715486 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.715470    2563 scope.go:117] "RemoveContainer" containerID="f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"
Jul 23 19:25:38.715724 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:25:38.715706    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc\": container with ID starting with f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc not found: ID does not exist" containerID="f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"
Jul 23 19:25:38.715782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.715732    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc"} err="failed to get container status \"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc\": rpc error: code = NotFound desc = could not find container \"f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc\": container with ID starting with f75ca12236791a148181563e427e6ade4143c31ddcd34476c6532d5195ed99bc not found: ID does not exist"
Jul 23 19:25:38.738853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.738832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:25:38.746506 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:38.746480    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-055d1f1e-kserve-bd6698787-66tdj"]
Jul 23 19:25:39.041414 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.041384    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:25:39.482394 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.482301    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" podUID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" containerName="main" containerID="cri-o://7926a2c791209c8fd4a48761cd64b58e8a62ff8b16cda1959d5d219985c35c25" gracePeriod=45
Jul 23 19:25:39.522098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.522072    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" path="/var/lib/kubelet/pods/3e8ece50-e577-45c6-8c48-d4d84d0ff902/volumes"
Jul 23 19:25:39.718736 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.718697    2563 generic.go:356] "Generic (PLEG): container finished" podID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" containerID="7926a2c791209c8fd4a48761cd64b58e8a62ff8b16cda1959d5d219985c35c25" exitCode=0
Jul 23 19:25:39.718903 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.718763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" event={"ID":"d54d082c-c636-40ba-bf81-ac99d3a5e9e7","Type":"ContainerDied","Data":"7926a2c791209c8fd4a48761cd64b58e8a62ff8b16cda1959d5d219985c35c25"}
Jul 23 19:25:39.745986 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.745930    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:39.834307 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834363    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834380    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834398    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834430    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzfv7\" (UniqueName: \"kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7\") pod \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\" (UID: \"d54d082c-c636-40ba-bf81-ac99d3a5e9e7\") "
Jul 23 19:25:39.834647 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834599    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:39.834755 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834645    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:39.834755 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.834652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home" (OuterVolumeSpecName: "home") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:39.836421 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.836396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm" (OuterVolumeSpecName: "dshm") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:25:39.836518 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.836483    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:25:39.836518 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.836502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7" (OuterVolumeSpecName: "kube-api-access-dzfv7") pod "d54d082c-c636-40ba-bf81-ac99d3a5e9e7" (UID: "d54d082c-c636-40ba-bf81-ac99d3a5e9e7"). InnerVolumeSpecName "kube-api-access-dzfv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:25:39.935921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935883    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:39.935921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935913    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:39.936092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935942    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:39.936092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935950    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:39.936092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935959    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzfv7\" (UniqueName: \"kubernetes.io/projected/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-kube-api-access-dzfv7\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:39.936092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:39.935969    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d54d082c-c636-40ba-bf81-ac99d3a5e9e7-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:25:40.725046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:40.725016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz" event={"ID":"d54d082c-c636-40ba-bf81-ac99d3a5e9e7","Type":"ContainerDied","Data":"17166e1b1ca479e3a3aab684815695ff0ca1cf664a2ba3d851ffbf7e7963aafc"}
Jul 23 19:25:40.725046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:40.725038    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"
Jul 23 19:25:40.725046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:40.725061    2563 scope.go:117] "RemoveContainer" containerID="7926a2c791209c8fd4a48761cd64b58e8a62ff8b16cda1959d5d219985c35c25"
Jul 23 19:25:40.747010 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:40.746987    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:40.751319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:40.751301    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-055d1f1e-kserve-5d9bc45c8c-srjrz"]
Jul 23 19:25:41.522336 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:41.522303    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" path="/var/lib/kubelet/pods/d54d082c-c636-40ba-bf81-ac99d3a5e9e7/volumes"
Jul 23 19:25:45.539449 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:25:45.539415    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:01.334227 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:01.334189    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:12.938577 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:12.938544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:22.433747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:22.433668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:32.823064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:32.823030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:41.830314 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:41.830276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:26:51.624829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:26:51.624780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:04.528743 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:04.528704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:13.628656 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:13.628620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:23.424487 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:23.424452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:33.135292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:33.135257    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:44.327051 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:44.327013    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:27:55.378243 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:27:55.378162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:28:07.447274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:28:07.447240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:28:19.951398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:28:19.951360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:28:32.828829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:28:32.828782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:28:45.843301 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:28:45.843267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:29:06.630020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:06.629967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:29:15.641422 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641392    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:15.641780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641740    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" containerName="main"
Jul 23 19:29:15.641780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641751    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" containerName="main"
Jul 23 19:29:15.641780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641767    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" containerName="main"
Jul 23 19:29:15.641780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641773    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" containerName="main"
Jul 23 19:29:15.641939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641853    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8ece50-e577-45c6-8c48-d4d84d0ff902" containerName="main"
Jul 23 19:29:15.641939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.641866    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d54d082c-c636-40ba-bf81-ac99d3a5e9e7" containerName="main"
Jul 23 19:29:15.644969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.644955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.647348 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.647323    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:29:15.647516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.647323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-de58cb5e-serving-cert\""
Jul 23 19:29:15.647516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.647335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:29:15.648028 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.648012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:29:15.654976 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.654951    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:15.689694 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.689668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.689845 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.689711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.689845 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.689749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f4j9\" (UniqueName: \"kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.790136 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.790101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.790292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.790141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.790292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.790175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f4j9\" (UniqueName: \"kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.790292 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:29:15.790270    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-serving-cert: secret "maas-api-e2e-ait-de58cb5e-serving-cert" not found
Jul 23 19:29:15.790448 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:29:15.790343    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls podName:e73ea763-2936-46fc-a380-c3dbcfd564eb nodeName:}" failed. No retries permitted until 2026-07-23 19:29:16.290322213 +0000 UTC m=+1699.192908272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls") pod "maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" (UID: "e73ea763-2936-46fc-a380-c3dbcfd564eb") : secret "maas-api-e2e-ait-de58cb5e-serving-cert" not found
Jul 23 19:29:15.790731 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.790712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:15.801964 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:15.801936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f4j9\" (UniqueName: \"kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:16.294701 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.294617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:16.297148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.297126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") pod \"maas-api-e2e-ait-de58cb5e-77855977c-kwkgp\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:16.555958 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.555875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:16.687736 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.687712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:16.689401 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:29:16.689376    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73ea763_2936_46fc_a380_c3dbcfd564eb.slice/crio-5777912f65497c7763e8f074370afa689eb2b4c0db1f8c9b485ab682444846a3 WatchSource:0}: Error finding container 5777912f65497c7763e8f074370afa689eb2b4c0db1f8c9b485ab682444846a3: Status 404 returned error can't find the container with id 5777912f65497c7763e8f074370afa689eb2b4c0db1f8c9b485ab682444846a3
Jul 23 19:29:16.690777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.690755    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:29:16.810359 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:16.810285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" event={"ID":"e73ea763-2936-46fc-a380-c3dbcfd564eb","Type":"ContainerStarted","Data":"5777912f65497c7763e8f074370afa689eb2b4c0db1f8c9b485ab682444846a3"}
Jul 23 19:29:19.828326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:19.828288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" event={"ID":"e73ea763-2936-46fc-a380-c3dbcfd564eb","Type":"ContainerStarted","Data":"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"}
Jul 23 19:29:19.828681 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:19.828377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:19.849514 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:19.849468    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" podStartSLOduration=2.43225751 podStartE2EDuration="4.84945489s" totalImagesPullingTime="2.41719738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:29:19.848011156 +0000 UTC m=+1702.750597245" watchObservedRunningTime="2026-07-23 19:29:19.84945489 +0000 UTC m=+1702.752040966"
Jul 23 19:29:22.884049 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:22.884015    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:29:25.839739 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:25.839714    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:41.089585 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.089552    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"]
Jul 23 19:29:41.092907 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.092890    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:41.103287 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.103266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:29:41.106296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.106274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"]
Jul 23 19:29:41.201948 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.201920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc\" (UID: \"dcff3f1f-49c3-4966-bd2d-ee676ed164e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:41.303096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.303072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc\" (UID: \"dcff3f1f-49c3-4966-bd2d-ee676ed164e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:41.303648 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.303612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc\" (UID: \"dcff3f1f-49c3-4966-bd2d-ee676ed164e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:41.402950 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.402890    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:41.537988 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.537962    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"]
Jul 23 19:29:41.539139 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:29:41.539117    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcff3f1f_49c3_4966_bd2d_ee676ed164e9.slice/crio-9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04 WatchSource:0}: Error finding container 9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04: Status 404 returned error can't find the container with id 9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04
Jul 23 19:29:41.942187 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:41.942153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc" event={"ID":"dcff3f1f-49c3-4966-bd2d-ee676ed164e9","Type":"ContainerStarted","Data":"9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04"}
Jul 23 19:29:44.958793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:44.958760    2563 generic.go:356] "Generic (PLEG): container finished" podID="dcff3f1f-49c3-4966-bd2d-ee676ed164e9" containerID="94c9ec94caf61e5e4a56938c388942660cc37a95e7bffc804e224ae9c9c304e8" exitCode=0
Jul 23 19:29:44.959169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:44.958855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc" event={"ID":"dcff3f1f-49c3-4966-bd2d-ee676ed164e9","Type":"ContainerDied","Data":"94c9ec94caf61e5e4a56938c388942660cc37a95e7bffc804e224ae9c9c304e8"}
Jul 23 19:29:46.095429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.095404    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:46.248528 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.248443    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca\") pod \"dcff3f1f-49c3-4966-bd2d-ee676ed164e9\" (UID: \"dcff3f1f-49c3-4966-bd2d-ee676ed164e9\") "
Jul 23 19:29:46.248799 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.248776    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dcff3f1f-49c3-4966-bd2d-ee676ed164e9" (UID: "dcff3f1f-49c3-4966-bd2d-ee676ed164e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:29:46.349198 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.349162    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcff3f1f-49c3-4966-bd2d-ee676ed164e9-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:29:46.970589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.970562    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"
Jul 23 19:29:46.970589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.970577    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc" event={"ID":"dcff3f1f-49c3-4966-bd2d-ee676ed164e9","Type":"ContainerDied","Data":"9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04"}
Jul 23 19:29:46.970794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:46.970619    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a6ce0823c1bdab467a0f2010a708e7340e88271b42a93f31f7a742ac0f38e04"
Jul 23 19:29:51.443199 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.443167    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:51.443540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.443402    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" podUID="e73ea763-2936-46fc-a380-c3dbcfd564eb" containerName="maas-api" containerID="cri-o://7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7" gracePeriod=30
Jul 23 19:29:51.689968 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.689946    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:51.794064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.794035    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") pod \"e73ea763-2936-46fc-a380-c3dbcfd564eb\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") "
Jul 23 19:29:51.794232 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.794112    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f4j9\" (UniqueName: \"kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9\") pod \"e73ea763-2936-46fc-a380-c3dbcfd564eb\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") "
Jul 23 19:29:51.794232 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.794146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca\") pod \"e73ea763-2936-46fc-a380-c3dbcfd564eb\" (UID: \"e73ea763-2936-46fc-a380-c3dbcfd564eb\") "
Jul 23 19:29:51.794502 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.794479    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e73ea763-2936-46fc-a380-c3dbcfd564eb" (UID: "e73ea763-2936-46fc-a380-c3dbcfd564eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:29:51.796246 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.796224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e73ea763-2936-46fc-a380-c3dbcfd564eb" (UID: "e73ea763-2936-46fc-a380-c3dbcfd564eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:29:51.796339 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.796313    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9" (OuterVolumeSpecName: "kube-api-access-4f4j9") pod "e73ea763-2936-46fc-a380-c3dbcfd564eb" (UID: "e73ea763-2936-46fc-a380-c3dbcfd564eb"). InnerVolumeSpecName "kube-api-access-4f4j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:29:51.895303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.895273    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e73ea763-2936-46fc-a380-c3dbcfd564eb-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:29:51.895303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.895299    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f4j9\" (UniqueName: \"kubernetes.io/projected/e73ea763-2936-46fc-a380-c3dbcfd564eb-kube-api-access-4f4j9\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:29:51.895303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.895309    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e73ea763-2936-46fc-a380-c3dbcfd564eb-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:29:51.998029 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.998000    2563 generic.go:356] "Generic (PLEG): container finished" podID="e73ea763-2936-46fc-a380-c3dbcfd564eb" containerID="7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7" exitCode=0
Jul 23 19:29:51.998154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.998045    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"
Jul 23 19:29:51.998154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.998061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" event={"ID":"e73ea763-2936-46fc-a380-c3dbcfd564eb","Type":"ContainerDied","Data":"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"}
Jul 23 19:29:51.998154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.998104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp" event={"ID":"e73ea763-2936-46fc-a380-c3dbcfd564eb","Type":"ContainerDied","Data":"5777912f65497c7763e8f074370afa689eb2b4c0db1f8c9b485ab682444846a3"}
Jul 23 19:29:51.998154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:51.998124    2563 scope.go:117] "RemoveContainer" containerID="7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"
Jul 23 19:29:52.007507 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:52.007492    2563 scope.go:117] "RemoveContainer" containerID="7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"
Jul 23 19:29:52.007796 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:29:52.007777    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7\": container with ID starting with 7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7 not found: ID does not exist" containerID="7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"
Jul 23 19:29:52.007879 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:52.007805    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7"} err="failed to get container status \"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7\": rpc error: code = NotFound desc = could not find container \"7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7\": container with ID starting with 7f56ebaa202bb5e1c0483aa2b83b5674a82628b6be47252b06a9c9f9c669cac7 not found: ID does not exist"
Jul 23 19:29:52.018719 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:52.018695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:52.022361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:52.022340    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de58cb5e-77855977c-kwkgp"]
Jul 23 19:29:53.521376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:53.521342    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e73ea763-2936-46fc-a380-c3dbcfd564eb" path="/var/lib/kubelet/pods/e73ea763-2936-46fc-a380-c3dbcfd564eb/volumes"
Jul 23 19:29:54.519012 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:54.518981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:29:56.130113 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:56.130082    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"]
Jul 23 19:29:56.132082 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:56.132061    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de58cb5e-rmrgc"]
Jul 23 19:29:57.523285 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:57.523249    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcff3f1f-49c3-4966-bd2d-ee676ed164e9" path="/var/lib/kubelet/pods/dcff3f1f-49c3-4966-bd2d-ee676ed164e9/volumes"
Jul 23 19:29:59.906290 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906249    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:29:59.906753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906738    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcff3f1f-49c3-4966-bd2d-ee676ed164e9" containerName="revoke-keys"
Jul 23 19:29:59.906832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906755    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcff3f1f-49c3-4966-bd2d-ee676ed164e9" containerName="revoke-keys"
Jul 23 19:29:59.906832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e73ea763-2936-46fc-a380-c3dbcfd564eb" containerName="maas-api"
Jul 23 19:29:59.906832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906802    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73ea763-2936-46fc-a380-c3dbcfd564eb" containerName="maas-api"
Jul 23 19:29:59.906997 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906891    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcff3f1f-49c3-4966-bd2d-ee676ed164e9" containerName="revoke-keys"
Jul 23 19:29:59.906997 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.906905    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e73ea763-2936-46fc-a380-c3dbcfd564eb" containerName="maas-api"
Jul 23 19:29:59.910250 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.910228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:29:59.912963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.912932    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:29:59.912963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.912936    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:29:59.913515 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.913497    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:29:59.913515 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.913511    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a0642743-serving-cert\""
Jul 23 19:29:59.919607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:29:59.919575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:30:00.059567 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.059537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.059567 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.059571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.059798 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.059688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8kvc\" (UniqueName: \"kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.134172 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.134145    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:00.137586 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.137571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"
Jul 23 19:30:00.139961 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.139939    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:30:00.145966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.145936    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:00.148281 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.148261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"
Jul 23 19:30:00.160516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.160492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.160696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.160672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.160850 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.160751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8kvc\" (UniqueName: \"kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.161472 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.161452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.163532 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.163512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.170457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.170435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8kvc\" (UniqueName: \"kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc\") pod \"maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.223746 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.223718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:00.302093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.302066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:00.302938 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:30:00.302913    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafebed5e_dbf3_4652_a51e_de4f13505740.slice/crio-8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b WatchSource:0}: Error finding container 8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b: Status 404 returned error can't find the container with id 8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b
Jul 23 19:30:00.369925 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:00.369896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:30:00.371169 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:30:00.371145    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d53818b_5fc6_4e02_97fa_e9c21e2f813f.slice/crio-2602b4eae25128472196d3785f08ee8190600b3bf4589f9f503244b4802ef082 WatchSource:0}: Error finding container 2602b4eae25128472196d3785f08ee8190600b3bf4589f9f503244b4802ef082: Status 404 returned error can't find the container with id 2602b4eae25128472196d3785f08ee8190600b3bf4589f9f503244b4802ef082
Jul 23 19:30:01.045490 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.045457    2563 generic.go:356] "Generic (PLEG): container finished" podID="afebed5e-dbf3-4652-a51e-de4f13505740" containerID="9c835147ac8cf0b7bf0d533409cdc2223f00e7bc14507871947ff21d3add85ad" exitCode=7
Jul 23 19:30:01.045873 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.045539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerDied","Data":"9c835147ac8cf0b7bf0d533409cdc2223f00e7bc14507871947ff21d3add85ad"}
Jul 23 19:30:01.045873 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.045630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerStarted","Data":"8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b"}
Jul 23 19:30:01.045873 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.045687    2563 scope.go:117] "RemoveContainer" containerID="9c835147ac8cf0b7bf0d533409cdc2223f00e7bc14507871947ff21d3add85ad"
Jul 23 19:30:01.047117 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.047096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" event={"ID":"1d53818b-5fc6-4e02-97fa-e9c21e2f813f","Type":"ContainerStarted","Data":"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"}
Jul 23 19:30:01.047238 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.047140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" event={"ID":"1d53818b-5fc6-4e02-97fa-e9c21e2f813f","Type":"ContainerStarted","Data":"2602b4eae25128472196d3785f08ee8190600b3bf4589f9f503244b4802ef082"}
Jul 23 19:30:01.047238 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.047188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:01.081020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:01.080978    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" podStartSLOduration=1.61285241 podStartE2EDuration="2.080964663s" totalImagesPullingTime="468.112253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:30:01.078832147 +0000 UTC m=+1743.981418218" watchObservedRunningTime="2026-07-23 19:30:01.080964663 +0000 UTC m=+1743.983550794"
Jul 23 19:30:02.053770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:02.053739    2563 generic.go:356] "Generic (PLEG): container finished" podID="afebed5e-dbf3-4652-a51e-de4f13505740" containerID="23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255" exitCode=7
Jul 23 19:30:02.054224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:02.053834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerDied","Data":"23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255"}
Jul 23 19:30:02.054224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:02.053891    2563 scope.go:117] "RemoveContainer" containerID="9c835147ac8cf0b7bf0d533409cdc2223f00e7bc14507871947ff21d3add85ad"
Jul 23 19:30:02.054224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:02.054068    2563 scope.go:117] "RemoveContainer" containerID="23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255"
Jul 23 19:30:02.054372 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:30:02.054282    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn_odh-ai-gateway-infra(afebed5e-dbf3-4652-a51e-de4f13505740)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" podUID="afebed5e-dbf3-4652-a51e-de4f13505740"
Jul 23 19:30:03.061325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:03.061299    2563 scope.go:117] "RemoveContainer" containerID="23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255"
Jul 23 19:30:03.061699 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:30:03.061517    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn_odh-ai-gateway-infra(afebed5e-dbf3-4652-a51e-de4f13505740)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" podUID="afebed5e-dbf3-4652-a51e-de4f13505740"
Jul 23 19:30:04.744424 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:04.744391    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:30:07.059126 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:07.059097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:16.516688 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:16.516656    2563 scope.go:117] "RemoveContainer" containerID="23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255"
Jul 23 19:30:17.135526 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:17.135494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerStarted","Data":"a492e4694066296c3fa59539b2cd583309c4f59471c87f61e0be2fd22461dbca"}
Jul 23 19:30:18.142004 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:18.141974    2563 generic.go:356] "Generic (PLEG): container finished" podID="afebed5e-dbf3-4652-a51e-de4f13505740" containerID="a492e4694066296c3fa59539b2cd583309c4f59471c87f61e0be2fd22461dbca" exitCode=0
Jul 23 19:30:18.142004 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:18.142007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerDied","Data":"a492e4694066296c3fa59539b2cd583309c4f59471c87f61e0be2fd22461dbca"}
Jul 23 19:30:18.142462 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:18.142053    2563 scope.go:117] "RemoveContainer" containerID="23a019387209e7f5a8ae78ad85d950dca2eb43c31d83dc39d3aa508572cbe255"
Jul 23 19:30:18.169433 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:18.169403    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:19.279474 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:19.279452    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"
Jul 23 19:30:20.156645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:20.156613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn" event={"ID":"afebed5e-dbf3-4652-a51e-de4f13505740","Type":"ContainerDied","Data":"8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b"}
Jul 23 19:30:20.156645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:20.156639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"
Jul 23 19:30:20.156887 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:20.156660    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e44cc9c7b79b974e4d07c629265ac4f5bb866eb1af14e41692b3e4f8b7b168b"
Jul 23 19:30:20.187034 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:20.187007    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:20.194712 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:20.194689    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-a0642743-29747250-r4fcn"]
Jul 23 19:30:21.522067 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:21.522033    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" path="/var/lib/kubelet/pods/afebed5e-dbf3-4652-a51e-de4f13505740/volumes"
Jul 23 19:30:24.858179 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858146    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"]
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858479    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858489    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858504    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858510    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858518    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858523    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858714 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858580    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.858714 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.858592    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:24.861886 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.861867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:24.864136 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.864121    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:30:24.877833 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.876212    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"]
Jul 23 19:30:24.946476 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:24.946449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a0642743-97tlm\" (UID: \"bd0221c3-04e5-48de-8722-3b71c1a338d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:25.047526 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:25.047496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a0642743-97tlm\" (UID: \"bd0221c3-04e5-48de-8722-3b71c1a338d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:25.048104 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:25.048088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a0642743-97tlm\" (UID: \"bd0221c3-04e5-48de-8722-3b71c1a338d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:25.173742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:25.173683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:25.303224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:25.303201    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"]
Jul 23 19:30:25.304606 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:30:25.304580    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0221c3_04e5_48de_8722_3b71c1a338d0.slice/crio-b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5 WatchSource:0}: Error finding container b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5: Status 404 returned error can't find the container with id b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5
Jul 23 19:30:26.187593 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:26.187552    2563 generic.go:356] "Generic (PLEG): container finished" podID="bd0221c3-04e5-48de-8722-3b71c1a338d0" containerID="31870766a1f21045948f567e41b58ae1cf95837e825da825262864faf22d3a2d" exitCode=0
Jul 23 19:30:26.188022 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:26.187607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm" event={"ID":"bd0221c3-04e5-48de-8722-3b71c1a338d0","Type":"ContainerDied","Data":"31870766a1f21045948f567e41b58ae1cf95837e825da825262864faf22d3a2d"}
Jul 23 19:30:26.188022 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:26.187678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm" event={"ID":"bd0221c3-04e5-48de-8722-3b71c1a338d0","Type":"ContainerStarted","Data":"b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5"}
Jul 23 19:30:27.330215 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:27.330192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:27.365224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:27.365202    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca\") pod \"bd0221c3-04e5-48de-8722-3b71c1a338d0\" (UID: \"bd0221c3-04e5-48de-8722-3b71c1a338d0\") "
Jul 23 19:30:27.365545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:27.365527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd0221c3-04e5-48de-8722-3b71c1a338d0" (UID: "bd0221c3-04e5-48de-8722-3b71c1a338d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:30:27.466118 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:27.466061    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0221c3-04e5-48de-8722-3b71c1a338d0-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:30:28.199273 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:28.199247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"
Jul 23 19:30:28.199475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:28.199283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm" event={"ID":"bd0221c3-04e5-48de-8722-3b71c1a338d0","Type":"ContainerDied","Data":"b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5"}
Jul 23 19:30:28.199475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:28.199327    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b93ef0d8ab091e034044e209ea7fb9f5de4e1722adeef367b7b5f1de04b43cf5"
Jul 23 19:30:35.385917 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.385880    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:30:35.386364 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.386084    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" podUID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" containerName="maas-api" containerID="cri-o://79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec" gracePeriod=30
Jul 23 19:30:35.636765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.636707    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:35.731371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.731341    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8kvc\" (UniqueName: \"kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc\") pod \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") "
Jul 23 19:30:35.731520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.731458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls\") pod \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") "
Jul 23 19:30:35.731520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.731484    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca\") pod \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\" (UID: \"1d53818b-5fc6-4e02-97fa-e9c21e2f813f\") "
Jul 23 19:30:35.731922 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.731894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d53818b-5fc6-4e02-97fa-e9c21e2f813f" (UID: "1d53818b-5fc6-4e02-97fa-e9c21e2f813f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:30:35.733541 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.733517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d53818b-5fc6-4e02-97fa-e9c21e2f813f" (UID: "1d53818b-5fc6-4e02-97fa-e9c21e2f813f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:30:35.733607 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.733577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc" (OuterVolumeSpecName: "kube-api-access-r8kvc") pod "1d53818b-5fc6-4e02-97fa-e9c21e2f813f" (UID: "1d53818b-5fc6-4e02-97fa-e9c21e2f813f"). InnerVolumeSpecName "kube-api-access-r8kvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:30:35.832830 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.832777    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:30:35.832830 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.832828    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:30:35.832830 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:35.832852    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8kvc\" (UniqueName: \"kubernetes.io/projected/1d53818b-5fc6-4e02-97fa-e9c21e2f813f-kube-api-access-r8kvc\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:30:36.242930 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.242899    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" containerID="79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec" exitCode=0
Jul 23 19:30:36.243130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.242943    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"
Jul 23 19:30:36.243130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.242989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" event={"ID":"1d53818b-5fc6-4e02-97fa-e9c21e2f813f","Type":"ContainerDied","Data":"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"}
Jul 23 19:30:36.243130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.243047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg" event={"ID":"1d53818b-5fc6-4e02-97fa-e9c21e2f813f","Type":"ContainerDied","Data":"2602b4eae25128472196d3785f08ee8190600b3bf4589f9f503244b4802ef082"}
Jul 23 19:30:36.243130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.243076    2563 scope.go:117] "RemoveContainer" containerID="79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"
Jul 23 19:30:36.252274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.252254    2563 scope.go:117] "RemoveContainer" containerID="79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"
Jul 23 19:30:36.252519 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:30:36.252501    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec\": container with ID starting with 79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec not found: ID does not exist" containerID="79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"
Jul 23 19:30:36.252587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.252526    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec"} err="failed to get container status \"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec\": rpc error: code = NotFound desc = could not find container \"79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec\": container with ID starting with 79e3696bad73994ac18f9c2646c8602058fe43b28f70ce761593211562da2cec not found: ID does not exist"
Jul 23 19:30:36.269366 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.269344    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:30:36.276029 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:36.276011    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a0642743-54c8f858d6-2mggg"]
Jul 23 19:30:37.521969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:37.521939    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" path="/var/lib/kubelet/pods/1d53818b-5fc6-4e02-97fa-e9c21e2f813f/volumes"
Jul 23 19:30:38.037320 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:38.037289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:30:39.942220 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:39.942159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"]
Jul 23 19:30:39.944671 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:39.944644    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a0642743-97tlm"]
Jul 23 19:30:41.522168 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:41.522133    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0221c3-04e5-48de-8722-3b71c1a338d0" path="/var/lib/kubelet/pods/bd0221c3-04e5-48de-8722-3b71c1a338d0/volumes"
Jul 23 19:30:43.217361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217330    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:30:43.217740 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217668    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0221c3-04e5-48de-8722-3b71c1a338d0" containerName="revoke-keys"
Jul 23 19:30:43.217740 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217681    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0221c3-04e5-48de-8722-3b71c1a338d0" containerName="revoke-keys"
Jul 23 19:30:43.217740 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217690    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" containerName="maas-api"
Jul 23 19:30:43.217740 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217695    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" containerName="maas-api"
Jul 23 19:30:43.217919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217745    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="afebed5e-dbf3-4652-a51e-de4f13505740" containerName="cleanup"
Jul 23 19:30:43.217919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217755    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0221c3-04e5-48de-8722-3b71c1a338d0" containerName="revoke-keys"
Jul 23 19:30:43.217919 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.217761    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d53818b-5fc6-4e02-97fa-e9c21e2f813f" containerName="maas-api"
Jul 23 19:30:43.222031 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.222011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.224898 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.224877    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:30:43.226765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.226746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:30:43.226875 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.226766    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1705afd4-serving-cert\""
Jul 23 19:30:43.226945 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.226880    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:30:43.236737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.236714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:30:43.293225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.293192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hh84\" (UniqueName: \"kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.293361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.293250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.293361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.293299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.394136 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.394099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.394311 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.394174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.394311 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.394228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hh84\" (UniqueName: \"kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.394311 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:30:43.394238    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-serving-cert: secret "maas-api-e2e-ait-1705afd4-serving-cert" not found
Jul 23 19:30:43.394311 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:30:43.394293    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls podName:531a8087-cecf-408a-8729-7f8965ac7ba9 nodeName:}" failed. No retries permitted until 2026-07-23 19:30:43.894276423 +0000 UTC m=+1786.796862478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls") pod "maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" (UID: "531a8087-cecf-408a-8729-7f8965ac7ba9") : secret "maas-api-e2e-ait-1705afd4-serving-cert" not found
Jul 23 19:30:43.394957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.394937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.410096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.410074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hh84\" (UniqueName: \"kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.898261 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.898233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:43.900489 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:43.900461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") pod \"maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:44.133739 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:44.133711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:44.267968 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:44.267938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:30:44.268762 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:30:44.268737    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531a8087_cecf_408a_8729_7f8965ac7ba9.slice/crio-6f9e7f409e0fcf44412e856fc702143df3f47b32954e9cf412ea4bc4c7d80bed WatchSource:0}: Error finding container 6f9e7f409e0fcf44412e856fc702143df3f47b32954e9cf412ea4bc4c7d80bed: Status 404 returned error can't find the container with id 6f9e7f409e0fcf44412e856fc702143df3f47b32954e9cf412ea4bc4c7d80bed
Jul 23 19:30:44.285992 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:44.285968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" event={"ID":"531a8087-cecf-408a-8729-7f8965ac7ba9","Type":"ContainerStarted","Data":"6f9e7f409e0fcf44412e856fc702143df3f47b32954e9cf412ea4bc4c7d80bed"}
Jul 23 19:30:45.298999 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:45.298967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" event={"ID":"531a8087-cecf-408a-8729-7f8965ac7ba9","Type":"ContainerStarted","Data":"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"}
Jul 23 19:30:45.299384 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:45.299090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:30:45.322044 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:45.321995    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" podStartSLOduration=1.8137749429999999 podStartE2EDuration="2.32198225s" totalImagesPullingTime="508.207307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:30:45.32101649 +0000 UTC m=+1788.223602579" watchObservedRunningTime="2026-07-23 19:30:45.32198225 +0000 UTC m=+1788.224568326"
Jul 23 19:30:49.437529 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:49.437491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:30:51.310394 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:30:51.310312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:31:08.925776 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:08.925735    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"]
Jul 23 19:31:08.929259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:08.929237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:08.931739 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:08.931723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:31:08.938429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:08.938404    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"]
Jul 23 19:31:09.006649 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.006612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb\" (UID: \"14c470f7-6124-4674-b01e-2992faae5c0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:09.107376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.107347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb\" (UID: \"14c470f7-6124-4674-b01e-2992faae5c0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:09.107975 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.107956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb\" (UID: \"14c470f7-6124-4674-b01e-2992faae5c0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:09.241799 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.241716    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:09.378363 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.376301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"]
Jul 23 19:31:09.380016 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:31:09.379988    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c470f7_6124_4674_b01e_2992faae5c0a.slice/crio-68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a WatchSource:0}: Error finding container 68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a: Status 404 returned error can't find the container with id 68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a
Jul 23 19:31:09.425827 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:09.425787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb" event={"ID":"14c470f7-6124-4674-b01e-2992faae5c0a","Type":"ContainerStarted","Data":"68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a"}
Jul 23 19:31:10.436698 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:10.436600    2563 generic.go:356] "Generic (PLEG): container finished" podID="14c470f7-6124-4674-b01e-2992faae5c0a" containerID="7d6ce438b3b58decd5befa9b7d63e9b2c8207c840fffdd004284d9d79d78d450" exitCode=0
Jul 23 19:31:10.436698 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:10.436634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb" event={"ID":"14c470f7-6124-4674-b01e-2992faae5c0a","Type":"ContainerDied","Data":"7d6ce438b3b58decd5befa9b7d63e9b2c8207c840fffdd004284d9d79d78d450"}
Jul 23 19:31:11.579343 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:11.579320    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:11.730722 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:11.730625    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca\") pod \"14c470f7-6124-4674-b01e-2992faae5c0a\" (UID: \"14c470f7-6124-4674-b01e-2992faae5c0a\") "
Jul 23 19:31:11.731013 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:11.730990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14c470f7-6124-4674-b01e-2992faae5c0a" (UID: "14c470f7-6124-4674-b01e-2992faae5c0a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:31:11.832339 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:11.832298    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14c470f7-6124-4674-b01e-2992faae5c0a-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:31:12.447998 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:12.447964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb" event={"ID":"14c470f7-6124-4674-b01e-2992faae5c0a","Type":"ContainerDied","Data":"68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a"}
Jul 23 19:31:12.448169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:12.448007    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"
Jul 23 19:31:12.448169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:12.448015    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68ed5d94a77b1ffeb6f1c00c9c68c7f0762ea77968a526374bee795cf54a817a"
Jul 23 19:31:19.472260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.472211    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:31:19.472756 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.472450    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" podUID="531a8087-cecf-408a-8729-7f8965ac7ba9" containerName="maas-api" containerID="cri-o://328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8" gracePeriod=30
Jul 23 19:31:19.720733 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.720705    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:31:19.896580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.896538    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca\") pod \"531a8087-cecf-408a-8729-7f8965ac7ba9\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") "
Jul 23 19:31:19.896789 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.896598    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") pod \"531a8087-cecf-408a-8729-7f8965ac7ba9\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") "
Jul 23 19:31:19.896789 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.896633    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hh84\" (UniqueName: \"kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84\") pod \"531a8087-cecf-408a-8729-7f8965ac7ba9\" (UID: \"531a8087-cecf-408a-8729-7f8965ac7ba9\") "
Jul 23 19:31:19.896946 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.896921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "531a8087-cecf-408a-8729-7f8965ac7ba9" (UID: "531a8087-cecf-408a-8729-7f8965ac7ba9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:31:19.898719 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.898689    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84" (OuterVolumeSpecName: "kube-api-access-7hh84") pod "531a8087-cecf-408a-8729-7f8965ac7ba9" (UID: "531a8087-cecf-408a-8729-7f8965ac7ba9"). InnerVolumeSpecName "kube-api-access-7hh84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:31:19.898850 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.898743    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "531a8087-cecf-408a-8729-7f8965ac7ba9" (UID: "531a8087-cecf-408a-8729-7f8965ac7ba9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:31:19.997737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.997696    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hh84\" (UniqueName: \"kubernetes.io/projected/531a8087-cecf-408a-8729-7f8965ac7ba9-kube-api-access-7hh84\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:31:19.997737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.997727    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531a8087-cecf-408a-8729-7f8965ac7ba9-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:31:19.997737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:19.997738    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531a8087-cecf-408a-8729-7f8965ac7ba9-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:31:20.492988 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.492948    2563 generic.go:356] "Generic (PLEG): container finished" podID="531a8087-cecf-408a-8729-7f8965ac7ba9" containerID="328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8" exitCode=0
Jul 23 19:31:20.493508 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.492995    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"
Jul 23 19:31:20.493508 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.493035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" event={"ID":"531a8087-cecf-408a-8729-7f8965ac7ba9","Type":"ContainerDied","Data":"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"}
Jul 23 19:31:20.493508 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.493089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf" event={"ID":"531a8087-cecf-408a-8729-7f8965ac7ba9","Type":"ContainerDied","Data":"6f9e7f409e0fcf44412e856fc702143df3f47b32954e9cf412ea4bc4c7d80bed"}
Jul 23 19:31:20.493508 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.493112    2563 scope.go:117] "RemoveContainer" containerID="328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"
Jul 23 19:31:20.502640 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.502621    2563 scope.go:117] "RemoveContainer" containerID="328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"
Jul 23 19:31:20.502934 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:31:20.502915    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8\": container with ID starting with 328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8 not found: ID does not exist" containerID="328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"
Jul 23 19:31:20.502986 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.502943    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8"} err="failed to get container status \"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8\": rpc error: code = NotFound desc = could not find container \"328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8\": container with ID starting with 328a4a0fde26a36e811a24c65da80eeeceec09c020ad47aa054c81e3b1f350b8 not found: ID does not exist"
Jul 23 19:31:20.516315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.516292    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:31:20.519861 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:20.519840    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1705afd4-7976866fb8-xqgbf"]
Jul 23 19:31:21.521977 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:21.521945    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="531a8087-cecf-408a-8729-7f8965ac7ba9" path="/var/lib/kubelet/pods/531a8087-cecf-408a-8729-7f8965ac7ba9/volumes"
Jul 23 19:31:22.527073 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:22.527033    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:31:23.995065 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:23.995031    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"]
Jul 23 19:31:23.998252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:23.998224    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1705afd4-w9qkb"]
Jul 23 19:31:25.521582 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:25.521550    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c470f7-6124-4674-b01e-2992faae5c0a" path="/var/lib/kubelet/pods/14c470f7-6124-4674-b01e-2992faae5c0a/volumes"
Jul 23 19:31:28.444197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444160    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:31:28.444579 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444540    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="531a8087-cecf-408a-8729-7f8965ac7ba9" containerName="maas-api"
Jul 23 19:31:28.444579 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444551    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="531a8087-cecf-408a-8729-7f8965ac7ba9" containerName="maas-api"
Jul 23 19:31:28.444579 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444560    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c470f7-6124-4674-b01e-2992faae5c0a" containerName="revoke-keys"
Jul 23 19:31:28.444579 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444566    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c470f7-6124-4674-b01e-2992faae5c0a" containerName="revoke-keys"
Jul 23 19:31:28.444735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444622    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="531a8087-cecf-408a-8729-7f8965ac7ba9" containerName="maas-api"
Jul 23 19:31:28.444735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.444630    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="14c470f7-6124-4674-b01e-2992faae5c0a" containerName="revoke-keys"
Jul 23 19:31:28.448939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.448923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.453266 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.453243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f36f30bf-serving-cert\""
Jul 23 19:31:28.453419 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.453318    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:31:28.453600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.453578    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:31:28.454121 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.454105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:31:28.463303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.463279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:31:28.571726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.571691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.571940 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.571772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.571940 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.571800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxdc\" (UniqueName: \"kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.672983 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.672946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.673154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.672990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxdc\" (UniqueName: \"kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.673154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.673032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.673687 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.673664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.675495 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.675472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.688223 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.688197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxdc\" (UniqueName: \"kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc\") pod \"maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.759182 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.759141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:28.897483 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:28.897458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:31:28.899765 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:31:28.899739    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0961a53f_ec76_49df_8d06_505df3d8651c.slice/crio-b068dd325f4918e29e6753fd2066d7426cd6f8eee61efe79184ef1bcb08efbff WatchSource:0}: Error finding container b068dd325f4918e29e6753fd2066d7426cd6f8eee61efe79184ef1bcb08efbff: Status 404 returned error can't find the container with id b068dd325f4918e29e6753fd2066d7426cd6f8eee61efe79184ef1bcb08efbff
Jul 23 19:31:29.542493 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:29.542457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" event={"ID":"0961a53f-ec76-49df-8d06-505df3d8651c","Type":"ContainerStarted","Data":"b068dd325f4918e29e6753fd2066d7426cd6f8eee61efe79184ef1bcb08efbff"}
Jul 23 19:31:30.553273 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:30.553241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" event={"ID":"0961a53f-ec76-49df-8d06-505df3d8651c","Type":"ContainerStarted","Data":"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"}
Jul 23 19:31:30.553657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:30.553360    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:30.576021 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:30.575972    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" podStartSLOduration=1.875543178 podStartE2EDuration="2.575957857s" totalImagesPullingTime="700.414679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:31:30.575191276 +0000 UTC m=+1833.477777355" watchObservedRunningTime="2026-07-23 19:31:30.575957857 +0000 UTC m=+1833.478543979"
Jul 23 19:31:33.328749 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:33.328708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:31:36.564056 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:36.564030    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:31:53.011769 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.011736    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"]
Jul 23 19:31:53.015371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.015354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:53.017782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.017761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:31:53.023078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.023051    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"]
Jul 23 19:31:53.066311 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.066270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4\" (UID: \"e8f75b57-f92f-462f-b851-002677ab8fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:53.167520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.167478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4\" (UID: \"e8f75b57-f92f-462f-b851-002677ab8fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:53.168124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.168106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4\" (UID: \"e8f75b57-f92f-462f-b851-002677ab8fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:53.329121 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.329016    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:53.468066 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.468030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"]
Jul 23 19:31:53.470443 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:31:53.470414    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f75b57_f92f_462f_b851_002677ab8fc7.slice/crio-07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382 WatchSource:0}: Error finding container 07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382: Status 404 returned error can't find the container with id 07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382
Jul 23 19:31:53.672129 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.672088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4" event={"ID":"e8f75b57-f92f-462f-b851-002677ab8fc7","Type":"ContainerStarted","Data":"85e2de5c4e55128c85503a31eea1c45383d9542bc8e77434a3906eb900ab4866"}
Jul 23 19:31:53.672328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.672147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4" event={"ID":"e8f75b57-f92f-462f-b851-002677ab8fc7","Type":"ContainerStarted","Data":"07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382"}
Jul 23 19:31:53.687604 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:53.687548    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4" podStartSLOduration=1.687532804 podStartE2EDuration="1.687532804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:31:53.68620223 +0000 UTC m=+1856.588788310" watchObservedRunningTime="2026-07-23 19:31:53.687532804 +0000 UTC m=+1856.590118883"
Jul 23 19:31:54.679298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:54.679262    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8f75b57-f92f-462f-b851-002677ab8fc7" containerID="85e2de5c4e55128c85503a31eea1c45383d9542bc8e77434a3906eb900ab4866" exitCode=0
Jul 23 19:31:54.679766 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:54.679354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4" event={"ID":"e8f75b57-f92f-462f-b851-002677ab8fc7","Type":"ContainerDied","Data":"85e2de5c4e55128c85503a31eea1c45383d9542bc8e77434a3906eb900ab4866"}
Jul 23 19:31:55.818461 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:55.818438    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:55.888121 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:55.888095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca\") pod \"e8f75b57-f92f-462f-b851-002677ab8fc7\" (UID: \"e8f75b57-f92f-462f-b851-002677ab8fc7\") "
Jul 23 19:31:55.888411 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:55.888389    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8f75b57-f92f-462f-b851-002677ab8fc7" (UID: "e8f75b57-f92f-462f-b851-002677ab8fc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:31:55.989093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:55.989023    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f75b57-f92f-462f-b851-002677ab8fc7-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:31:56.690597 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:56.690567    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"
Jul 23 19:31:56.690769 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:56.690609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4" event={"ID":"e8f75b57-f92f-462f-b851-002677ab8fc7","Type":"ContainerDied","Data":"07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382"}
Jul 23 19:31:56.690769 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:31:56.690656    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07d16886b22bb34121b39c651ce73d817ee5401030c2e1bc3220a33e7e794382"
Jul 23 19:32:03.433662 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.433622    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:32:03.434138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.433917    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" podUID="0961a53f-ec76-49df-8d06-505df3d8651c" containerName="maas-api" containerID="cri-o://3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a" gracePeriod=30
Jul 23 19:32:03.684589 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.684534    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:32:03.727179 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.727152    2563 generic.go:356] "Generic (PLEG): container finished" podID="0961a53f-ec76-49df-8d06-505df3d8651c" containerID="3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a" exitCode=0
Jul 23 19:32:03.727328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.727191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" event={"ID":"0961a53f-ec76-49df-8d06-505df3d8651c","Type":"ContainerDied","Data":"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"}
Jul 23 19:32:03.727328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.727248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k" event={"ID":"0961a53f-ec76-49df-8d06-505df3d8651c","Type":"ContainerDied","Data":"b068dd325f4918e29e6753fd2066d7426cd6f8eee61efe79184ef1bcb08efbff"}
Jul 23 19:32:03.727328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.727209    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"
Jul 23 19:32:03.727328 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.727273    2563 scope.go:117] "RemoveContainer" containerID="3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"
Jul 23 19:32:03.737414 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.737386    2563 scope.go:117] "RemoveContainer" containerID="3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"
Jul 23 19:32:03.737675 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:32:03.737655    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a\": container with ID starting with 3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a not found: ID does not exist" containerID="3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"
Jul 23 19:32:03.737736 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.737683    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a"} err="failed to get container status \"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a\": rpc error: code = NotFound desc = could not find container \"3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a\": container with ID starting with 3d97e1e50cec14678683fdaebf031a25acd75aeef3d3bc669cf97c40ca5d055a not found: ID does not exist"
Jul 23 19:32:03.749050 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.749029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls\") pod \"0961a53f-ec76-49df-8d06-505df3d8651c\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") "
Jul 23 19:32:03.749124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.749056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca\") pod \"0961a53f-ec76-49df-8d06-505df3d8651c\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") "
Jul 23 19:32:03.749124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.749072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgxdc\" (UniqueName: \"kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc\") pod \"0961a53f-ec76-49df-8d06-505df3d8651c\" (UID: \"0961a53f-ec76-49df-8d06-505df3d8651c\") "
Jul 23 19:32:03.749376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.749354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0961a53f-ec76-49df-8d06-505df3d8651c" (UID: "0961a53f-ec76-49df-8d06-505df3d8651c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:32:03.751005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.750982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0961a53f-ec76-49df-8d06-505df3d8651c" (UID: "0961a53f-ec76-49df-8d06-505df3d8651c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:32:03.751115 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.751048    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc" (OuterVolumeSpecName: "kube-api-access-tgxdc") pod "0961a53f-ec76-49df-8d06-505df3d8651c" (UID: "0961a53f-ec76-49df-8d06-505df3d8651c"). InnerVolumeSpecName "kube-api-access-tgxdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:32:03.850102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.850072    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0961a53f-ec76-49df-8d06-505df3d8651c-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:32:03.850102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.850100    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0961a53f-ec76-49df-8d06-505df3d8651c-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:32:03.850102 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:03.850112    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgxdc\" (UniqueName: \"kubernetes.io/projected/0961a53f-ec76-49df-8d06-505df3d8651c-kube-api-access-tgxdc\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:32:04.054479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:04.054449    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:32:04.058641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:04.058613    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f36f30bf-5f54d9d64f-zwq7k"]
Jul 23 19:32:05.521622 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:05.521589    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0961a53f-ec76-49df-8d06-505df3d8651c" path="/var/lib/kubelet/pods/0961a53f-ec76-49df-8d06-505df3d8651c/volumes"
Jul 23 19:32:06.627362 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:06.627330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:32:08.079342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:08.079302    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"]
Jul 23 19:32:08.081333 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:08.081311    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f36f30bf-6q4z4"]
Jul 23 19:32:09.521615 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:32:09.521581    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8f75b57-f92f-462f-b851-002677ab8fc7" path="/var/lib/kubelet/pods/e8f75b57-f92f-462f-b851-002677ab8fc7/volumes"
Jul 23 19:33:35.933650 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:35.933612    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:33:48.123218 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:48.123181    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:33:51.544138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544054    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:33:51.544591 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544566    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0961a53f-ec76-49df-8d06-505df3d8651c" containerName="maas-api"
Jul 23 19:33:51.544591 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544584    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0961a53f-ec76-49df-8d06-505df3d8651c" containerName="maas-api"
Jul 23 19:33:51.544711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8f75b57-f92f-462f-b851-002677ab8fc7" containerName="revoke-keys"
Jul 23 19:33:51.544711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544625    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f75b57-f92f-462f-b851-002677ab8fc7" containerName="revoke-keys"
Jul 23 19:33:51.544832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544723    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8f75b57-f92f-462f-b851-002677ab8fc7" containerName="revoke-keys"
Jul 23 19:33:51.544832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.544741    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0961a53f-ec76-49df-8d06-505df3d8651c" containerName="maas-api"
Jul 23 19:33:51.548005 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.547984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.550468 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.550446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4965db3e-serving-cert\""
Jul 23 19:33:51.550594 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.550481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:33:51.550594 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.550525    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:33:51.550780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.550755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:33:51.560168 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.560145    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:33:51.737534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.737501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.737735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.737559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.737735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.737627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8p9q\" (UniqueName: \"kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.839156 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.839071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8p9q\" (UniqueName: \"kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.839156 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.839124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.839383 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.839162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.839907 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.839881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.841742 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.841710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.847904 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.847874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8p9q\" (UniqueName: \"kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q\") pod \"maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:51.861706 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:51.861683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:52.004416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:52.004383    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:33:52.005632 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:33:52.005608    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dee2af7_1e67_46b1_b8c2_527007c313f3.slice/crio-fefc4525764bac23694a27aa5146f31425a336517cf904330accbb719006ff2f WatchSource:0}: Error finding container fefc4525764bac23694a27aa5146f31425a336517cf904330accbb719006ff2f: Status 404 returned error can't find the container with id fefc4525764bac23694a27aa5146f31425a336517cf904330accbb719006ff2f
Jul 23 19:33:52.331349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:52.331316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" event={"ID":"2dee2af7-1e67-46b1-b8c2-527007c313f3","Type":"ContainerStarted","Data":"fefc4525764bac23694a27aa5146f31425a336517cf904330accbb719006ff2f"}
Jul 23 19:33:53.338387 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:53.338352    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" event={"ID":"2dee2af7-1e67-46b1-b8c2-527007c313f3","Type":"ContainerStarted","Data":"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"}
Jul 23 19:33:53.338833 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:53.338419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:33:53.355471 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:53.355425    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" podStartSLOduration=1.601870401 podStartE2EDuration="2.355410186s" totalImagesPullingTime="753.539785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:33:53.354884853 +0000 UTC m=+1976.257470931" watchObservedRunningTime="2026-07-23 19:33:53.355410186 +0000 UTC m=+1976.257996283"
Jul 23 19:33:59.350120 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:33:59.350090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:34:01.323869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:01.323799    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:34:03.459576 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.459533    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"]
Jul 23 19:34:03.463644 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.463620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:03.465936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.465910    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:34:03.473771 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.473743    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"]
Jul 23 19:34:03.537962 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.537924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4965db3e-67rjk\" (UID: \"3a94847d-4288-430b-9996-58eb0c4075d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:03.638510 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.638467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4965db3e-67rjk\" (UID: \"3a94847d-4288-430b-9996-58eb0c4075d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:03.639148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.639130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4965db3e-67rjk\" (UID: \"3a94847d-4288-430b-9996-58eb0c4075d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:03.776695 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.776660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:03.913954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:03.913920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"]
Jul 23 19:34:03.915169 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:34:03.915141    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a94847d_4288_430b_9996_58eb0c4075d7.slice/crio-25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898 WatchSource:0}: Error finding container 25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898: Status 404 returned error can't find the container with id 25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898
Jul 23 19:34:04.400105 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:04.400013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk" event={"ID":"3a94847d-4288-430b-9996-58eb0c4075d7","Type":"ContainerStarted","Data":"ed50a907928b59310c986860b3e7514062a95a16234552291d236c999f7aee9e"}
Jul 23 19:34:04.400105 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:04.400070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk" event={"ID":"3a94847d-4288-430b-9996-58eb0c4075d7","Type":"ContainerStarted","Data":"25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898"}
Jul 23 19:34:04.416858 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:04.416796    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk" podStartSLOduration=1.416780401 podStartE2EDuration="1.416780401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:34:04.415483195 +0000 UTC m=+1987.318069273" watchObservedRunningTime="2026-07-23 19:34:04.416780401 +0000 UTC m=+1987.319366479"
Jul 23 19:34:05.406274 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:05.406239    2563 generic.go:356] "Generic (PLEG): container finished" podID="3a94847d-4288-430b-9996-58eb0c4075d7" containerID="ed50a907928b59310c986860b3e7514062a95a16234552291d236c999f7aee9e" exitCode=0
Jul 23 19:34:05.406653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:05.406313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk" event={"ID":"3a94847d-4288-430b-9996-58eb0c4075d7","Type":"ContainerDied","Data":"ed50a907928b59310c986860b3e7514062a95a16234552291d236c999f7aee9e"}
Jul 23 19:34:06.555765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:06.555736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:06.665103 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:06.665019    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca\") pod \"3a94847d-4288-430b-9996-58eb0c4075d7\" (UID: \"3a94847d-4288-430b-9996-58eb0c4075d7\") "
Jul 23 19:34:06.665425 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:06.665403    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a94847d-4288-430b-9996-58eb0c4075d7" (UID: "3a94847d-4288-430b-9996-58eb0c4075d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:34:06.765836 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:06.765787    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a94847d-4288-430b-9996-58eb0c4075d7-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:34:07.418772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:07.418738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk" event={"ID":"3a94847d-4288-430b-9996-58eb0c4075d7","Type":"ContainerDied","Data":"25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898"}
Jul 23 19:34:07.418772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:07.418760    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"
Jul 23 19:34:07.419089 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:07.418829    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25968cacb7708869d49e903d63445f0ad9da90b006610c590d845263adf65898"
Jul 23 19:34:13.852929 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:13.852896    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:34:13.853345 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:13.853197    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" podUID="2dee2af7-1e67-46b1-b8c2-527007c313f3" containerName="maas-api" containerID="cri-o://14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e" gracePeriod=30
Jul 23 19:34:14.106040 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.105974    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:34:14.227577 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.227539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca\") pod \"2dee2af7-1e67-46b1-b8c2-527007c313f3\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") "
Jul 23 19:34:14.227753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.227594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8p9q\" (UniqueName: \"kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q\") pod \"2dee2af7-1e67-46b1-b8c2-527007c313f3\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") "
Jul 23 19:34:14.227753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.227650    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls\") pod \"2dee2af7-1e67-46b1-b8c2-527007c313f3\" (UID: \"2dee2af7-1e67-46b1-b8c2-527007c313f3\") "
Jul 23 19:34:14.228000 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.227981    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2dee2af7-1e67-46b1-b8c2-527007c313f3" (UID: "2dee2af7-1e67-46b1-b8c2-527007c313f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:34:14.229765 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.229738    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q" (OuterVolumeSpecName: "kube-api-access-j8p9q") pod "2dee2af7-1e67-46b1-b8c2-527007c313f3" (UID: "2dee2af7-1e67-46b1-b8c2-527007c313f3"). InnerVolumeSpecName "kube-api-access-j8p9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:34:14.229864 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.229767    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2dee2af7-1e67-46b1-b8c2-527007c313f3" (UID: "2dee2af7-1e67-46b1-b8c2-527007c313f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:34:14.328349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.328304    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dee2af7-1e67-46b1-b8c2-527007c313f3-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:34:14.328349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.328344    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8p9q\" (UniqueName: \"kubernetes.io/projected/2dee2af7-1e67-46b1-b8c2-527007c313f3-kube-api-access-j8p9q\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:34:14.328349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.328355    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dee2af7-1e67-46b1-b8c2-527007c313f3-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:34:14.458583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.458493    2563 generic.go:356] "Generic (PLEG): container finished" podID="2dee2af7-1e67-46b1-b8c2-527007c313f3" containerID="14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e" exitCode=0
Jul 23 19:34:14.458583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.458532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" event={"ID":"2dee2af7-1e67-46b1-b8c2-527007c313f3","Type":"ContainerDied","Data":"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"}
Jul 23 19:34:14.458583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.458569    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"
Jul 23 19:34:14.458837 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.458601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92" event={"ID":"2dee2af7-1e67-46b1-b8c2-527007c313f3","Type":"ContainerDied","Data":"fefc4525764bac23694a27aa5146f31425a336517cf904330accbb719006ff2f"}
Jul 23 19:34:14.458837 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.458626    2563 scope.go:117] "RemoveContainer" containerID="14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"
Jul 23 19:34:14.468994 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.468975    2563 scope.go:117] "RemoveContainer" containerID="14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"
Jul 23 19:34:14.469271 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:34:14.469253    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e\": container with ID starting with 14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e not found: ID does not exist" containerID="14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"
Jul 23 19:34:14.469327 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.469279    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e"} err="failed to get container status \"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e\": rpc error: code = NotFound desc = could not find container \"14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e\": container with ID starting with 14443f2744245280d20fd2574424447257fcb7d81e26675fe2634919d858959e not found: ID does not exist"
Jul 23 19:34:14.482359 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.482333    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:34:14.486528 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:14.486501    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4965db3e-59b9b4f9b7-5xk92"]
Jul 23 19:34:15.523008 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:15.522977    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dee2af7-1e67-46b1-b8c2-527007c313f3" path="/var/lib/kubelet/pods/2dee2af7-1e67-46b1-b8c2-527007c313f3/volumes"
Jul 23 19:34:17.189138 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:17.189100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:34:18.542953 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:18.542921    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"]
Jul 23 19:34:18.544969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:18.544948    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4965db3e-67rjk"]
Jul 23 19:34:19.521524 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:19.521490    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a94847d-4288-430b-9996-58eb0c4075d7" path="/var/lib/kubelet/pods/3a94847d-4288-430b-9996-58eb0c4075d7/volumes"
Jul 23 19:34:47.643753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.643694    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644244    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dee2af7-1e67-46b1-b8c2-527007c313f3" containerName="maas-api"
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644261    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dee2af7-1e67-46b1-b8c2-527007c313f3" containerName="maas-api"
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644273    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a94847d-4288-430b-9996-58eb0c4075d7" containerName="revoke-keys"
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644282    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a94847d-4288-430b-9996-58eb0c4075d7" containerName="revoke-keys"
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644368    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dee2af7-1e67-46b1-b8c2-527007c313f3" containerName="maas-api"
Jul 23 19:34:47.644380 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.644377    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a94847d-4288-430b-9996-58eb0c4075d7" containerName="revoke-keys"
Jul 23 19:34:47.647804 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.647782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.650216 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.650190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5ccd9e4a-serving-cert\""
Jul 23 19:34:47.650361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.650224    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:34:47.650361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.650327    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:34:47.650934 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.650916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:34:47.656307 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.656281    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:34:47.722617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.722573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5n2\" (UniqueName: \"kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.722795 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.722667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.722795 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.722738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.823391 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.823346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.823570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.823425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.823570 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.823487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5n2\" (UniqueName: \"kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.823570 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:34:47.823490    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-serving-cert: secret "maas-api-e2e-mt-5ccd9e4a-serving-cert" not found
Jul 23 19:34:47.823699 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:34:47.823571    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls podName:4706565c-dc19-4e6b-bd0d-0353cc5ef285 nodeName:}" failed. No retries permitted until 2026-07-23 19:34:48.323549258 +0000 UTC m=+2031.226135314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls") pod "maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" (UID: "4706565c-dc19-4e6b-bd0d-0353cc5ef285") : secret "maas-api-e2e-mt-5ccd9e4a-serving-cert" not found
Jul 23 19:34:47.824091 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.824073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:47.832697 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:47.832671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5n2\" (UniqueName: \"kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:48.328118 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:48.328072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:48.330641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:48.330614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") pod \"maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:48.561990 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:48.561955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:48.701667 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:48.701641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:34:48.703251 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:34:48.703221    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4706565c_dc19_4e6b_bd0d_0353cc5ef285.slice/crio-d42b193ebd75939aa0688b6e0686655e67e4ccafebb5f06b5115c964ccbe9b6b WatchSource:0}: Error finding container d42b193ebd75939aa0688b6e0686655e67e4ccafebb5f06b5115c964ccbe9b6b: Status 404 returned error can't find the container with id d42b193ebd75939aa0688b6e0686655e67e4ccafebb5f06b5115c964ccbe9b6b
Jul 23 19:34:48.704596 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:48.704578    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:34:49.659790 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:49.659754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" event={"ID":"4706565c-dc19-4e6b-bd0d-0353cc5ef285","Type":"ContainerStarted","Data":"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"}
Jul 23 19:34:49.659994 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:49.659806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" event={"ID":"4706565c-dc19-4e6b-bd0d-0353cc5ef285","Type":"ContainerStarted","Data":"d42b193ebd75939aa0688b6e0686655e67e4ccafebb5f06b5115c964ccbe9b6b"}
Jul 23 19:34:49.659994 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:49.659859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:49.680913 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:49.680861    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" podStartSLOduration=2.25271999 podStartE2EDuration="2.680846452s" totalImagesPullingTime="428.126462ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:34:49.677916327 +0000 UTC m=+2032.580502434" watchObservedRunningTime="2026-07-23 19:34:49.680846452 +0000 UTC m=+2032.583432531"
Jul 23 19:34:53.028548 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:53.028514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:34:55.670967 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:55.670937    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:34:57.472434 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.472397    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:34:57.476132 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.476113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.478432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.478407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5ccd9e4a\"/\"kube-root-ca.crt\""
Jul 23 19:34:57.479211 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.479188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5ccd9e4a\"/\"e2e-lifecycle-model-5ccd9e4a-kserve-self-signed-certs\""
Jul 23 19:34:57.479343 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.479314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5ccd9e4a\"/\"default-dockercfg-7lcvw\""
Jul 23 19:34:57.479457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.479418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5ccd9e4a\"/\"openshift-service-ca.crt\""
Jul 23 19:34:57.486678 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.486658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:34:57.508371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.508535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.508535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.508535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.508535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.508535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.508505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxb2j\" (UniqueName: \"kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609229 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxb2j\" (UniqueName: \"kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609729 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609838 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.609907 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.609885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.611735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.611702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.612349 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.612327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.617340 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.617318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxb2j\" (UniqueName: \"kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j\") pod \"e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") " pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.785891 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.785847    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:34:57.928455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:57.928425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:34:57.930122 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:34:57.930094    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7165c95_33e2_43cf_ba01_14d624caa1b8.slice/crio-7bdae3531700c89b676a11f9b110fae78315b3e086ab4376861fae35d4cc25a5 WatchSource:0}: Error finding container 7bdae3531700c89b676a11f9b110fae78315b3e086ab4376861fae35d4cc25a5: Status 404 returned error can't find the container with id 7bdae3531700c89b676a11f9b110fae78315b3e086ab4376861fae35d4cc25a5
Jul 23 19:34:58.716426 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:58.716342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" event={"ID":"d7165c95-33e2-43cf-ba01-14d624caa1b8","Type":"ContainerStarted","Data":"73b1ae6be58a7ca5701780de26c28df55ae90ac1e8ec8986111ec5e3b95dd201"}
Jul 23 19:34:58.716426 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:58.716384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" event={"ID":"d7165c95-33e2-43cf-ba01-14d624caa1b8","Type":"ContainerStarted","Data":"7bdae3531700c89b676a11f9b110fae78315b3e086ab4376861fae35d4cc25a5"}
Jul 23 19:34:58.733563 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:34:58.733512    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" podStartSLOduration=1.733496402 podStartE2EDuration="1.733496402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:34:58.732443023 +0000 UTC m=+2041.635029164" watchObservedRunningTime="2026-07-23 19:34:58.733496402 +0000 UTC m=+2041.636082517"
Jul 23 19:35:01.229620 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:01.229582    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:35:07.786520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:07.786486    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:35:07.787003 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:07.786536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:35:07.799224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:07.799202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:35:08.781635 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:08.781596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:35:19.228756 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:19.228711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:35:39.427061 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:39.426983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:35:40.972368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:40.972335    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:35:40.972803 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:40.972577    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" podUID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" containerName="maas-api" containerID="cri-o://0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb" gracePeriod=30
Jul 23 19:35:41.224091 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.224037    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:35:41.298442 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.298411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st5n2\" (UniqueName: \"kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2\") pod \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") "
Jul 23 19:35:41.298587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.298478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca\") pod \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") "
Jul 23 19:35:41.298587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.298539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") pod \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\" (UID: \"4706565c-dc19-4e6b-bd0d-0353cc5ef285\") "
Jul 23 19:35:41.298808 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.298782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4706565c-dc19-4e6b-bd0d-0353cc5ef285" (UID: "4706565c-dc19-4e6b-bd0d-0353cc5ef285"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:35:41.300502 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.300480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2" (OuterVolumeSpecName: "kube-api-access-st5n2") pod "4706565c-dc19-4e6b-bd0d-0353cc5ef285" (UID: "4706565c-dc19-4e6b-bd0d-0353cc5ef285"). InnerVolumeSpecName "kube-api-access-st5n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:35:41.300567 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.300506    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4706565c-dc19-4e6b-bd0d-0353cc5ef285" (UID: "4706565c-dc19-4e6b-bd0d-0353cc5ef285"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:35:41.399979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.399954    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4706565c-dc19-4e6b-bd0d-0353cc5ef285-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:35:41.399979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.399976    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st5n2\" (UniqueName: \"kubernetes.io/projected/4706565c-dc19-4e6b-bd0d-0353cc5ef285-kube-api-access-st5n2\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:35:41.400137 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.399986    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4706565c-dc19-4e6b-bd0d-0353cc5ef285-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:35:41.949869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.949791    2563 generic.go:356] "Generic (PLEG): container finished" podID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" containerID="0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb" exitCode=0
Jul 23 19:35:41.949869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.949851    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"
Jul 23 19:35:41.950046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.949874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" event={"ID":"4706565c-dc19-4e6b-bd0d-0353cc5ef285","Type":"ContainerDied","Data":"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"}
Jul 23 19:35:41.950046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.949929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7" event={"ID":"4706565c-dc19-4e6b-bd0d-0353cc5ef285","Type":"ContainerDied","Data":"d42b193ebd75939aa0688b6e0686655e67e4ccafebb5f06b5115c964ccbe9b6b"}
Jul 23 19:35:41.950046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.949952    2563 scope.go:117] "RemoveContainer" containerID="0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"
Jul 23 19:35:41.959194 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.959178    2563 scope.go:117] "RemoveContainer" containerID="0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"
Jul 23 19:35:41.959434 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:35:41.959418    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb\": container with ID starting with 0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb not found: ID does not exist" containerID="0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"
Jul 23 19:35:41.959489 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.959441    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb"} err="failed to get container status \"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb\": rpc error: code = NotFound desc = could not find container \"0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb\": container with ID starting with 0f6de6f71c171e9d502d133b00587ebacccea39c26f7accce22af218ab884deb not found: ID does not exist"
Jul 23 19:35:41.966258 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.966236    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:35:41.970173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:41.970154    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5ccd9e4a-58d4f56c65-mtgl7"]
Jul 23 19:35:43.522219 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:43.522185    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" path="/var/lib/kubelet/pods/4706565c-dc19-4e6b-bd0d-0353cc5ef285/volumes"
Jul 23 19:35:49.028246 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:49.028205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:35:52.848807 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:52.848768    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:35:57.694006 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:35:57.693971    2563 scope.go:117] "RemoveContainer" containerID="94c9ec94caf61e5e4a56938c388942660cc37a95e7bffc804e224ae9c9c304e8"
Jul 23 19:36:07.869309 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:07.869246    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" podUID="d7165c95-33e2-43cf-ba01-14d624caa1b8" containerName="main" containerID="cri-o://73b1ae6be58a7ca5701780de26c28df55ae90ac1e8ec8986111ec5e3b95dd201" gracePeriod=45
Jul 23 19:36:08.097869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.097840    2563 generic.go:356] "Generic (PLEG): container finished" podID="d7165c95-33e2-43cf-ba01-14d624caa1b8" containerID="73b1ae6be58a7ca5701780de26c28df55ae90ac1e8ec8986111ec5e3b95dd201" exitCode=0
Jul 23 19:36:08.098020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.097903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" event={"ID":"d7165c95-33e2-43cf-ba01-14d624caa1b8","Type":"ContainerDied","Data":"73b1ae6be58a7ca5701780de26c28df55ae90ac1e8ec8986111ec5e3b95dd201"}
Jul 23 19:36:08.122847 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.122776    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:36:08.235744 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.235914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.235914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235790    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxb2j\" (UniqueName: \"kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.235914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.235914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.236156 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.235924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir\") pod \"d7165c95-33e2-43cf-ba01-14d624caa1b8\" (UID: \"d7165c95-33e2-43cf-ba01-14d624caa1b8\") "
Jul 23 19:36:08.236225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.236132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:36:08.236225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.236169    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home" (OuterVolumeSpecName: "home") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:36:08.236225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.236196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:36:08.238023 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.238005    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm" (OuterVolumeSpecName: "dshm") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:36:08.238262 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.238036    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j" (OuterVolumeSpecName: "kube-api-access-fxb2j") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "kube-api-access-fxb2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:36:08.238262 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.238097    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7165c95-33e2-43cf-ba01-14d624caa1b8" (UID: "d7165c95-33e2-43cf-ba01-14d624caa1b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:36:08.337203 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337173    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:08.337203 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337197    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7165c95-33e2-43cf-ba01-14d624caa1b8-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:08.337203 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337207    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:08.337396 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337216    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxb2j\" (UniqueName: \"kubernetes.io/projected/d7165c95-33e2-43cf-ba01-14d624caa1b8-kube-api-access-fxb2j\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:08.337396 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337225    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:08.337396 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:08.337233    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7165c95-33e2-43cf-ba01-14d624caa1b8-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:36:09.107078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.107049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6" event={"ID":"d7165c95-33e2-43cf-ba01-14d624caa1b8","Type":"ContainerDied","Data":"7bdae3531700c89b676a11f9b110fae78315b3e086ab4376861fae35d4cc25a5"}
Jul 23 19:36:09.107078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.107071    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"
Jul 23 19:36:09.107078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.107093    2563 scope.go:117] "RemoveContainer" containerID="73b1ae6be58a7ca5701780de26c28df55ae90ac1e8ec8986111ec5e3b95dd201"
Jul 23 19:36:09.129550 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.129528    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:36:09.133384 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.133361    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5ccd9e4a/e2e-lifecycle-model-5ccd9e4a-kserve-54cb779b9f-g92m6"]
Jul 23 19:36:09.521553 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:09.521523    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7165c95-33e2-43cf-ba01-14d624caa1b8" path="/var/lib/kubelet/pods/d7165c95-33e2-43cf-ba01-14d624caa1b8/volumes"
Jul 23 19:36:40.479704 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:40.479673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:36:49.223446 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:49.223408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:36:57.719160 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:57.719092    2563 scope.go:117] "RemoveContainer" containerID="31870766a1f21045948f567e41b58ae1cf95837e825da825262864faf22d3a2d"
Jul 23 19:36:57.727507 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:36:57.727486    2563 scope.go:117] "RemoveContainer" containerID="a492e4694066296c3fa59539b2cd583309c4f59471c87f61e0be2fd22461dbca"
Jul 23 19:37:36.526849 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.526798    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527369    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" containerName="maas-api"
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527385    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" containerName="maas-api"
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527422    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7165c95-33e2-43cf-ba01-14d624caa1b8" containerName="main"
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527431    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7165c95-33e2-43cf-ba01-14d624caa1b8" containerName="main"
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527511    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7165c95-33e2-43cf-ba01-14d624caa1b8" containerName="main"
Jul 23 19:37:36.529305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.527523    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4706565c-dc19-4e6b-bd0d-0353cc5ef285" containerName="maas-api"
Jul 23 19:37:36.530915 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.530884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.533457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.533437    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:37:36.533457 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.533446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:37:36.533639 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.533487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-453525-serving-cert\""
Jul 23 19:37:36.533639 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.533451    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:37:36.541626 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.541604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:37:36.558830 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.558790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwdc5\" (UniqueName: \"kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.558966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.558845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.558966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.558889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.660381 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.660347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwdc5\" (UniqueName: \"kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.660563 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.660390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.660563 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.660520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.660690 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:37:36.660641    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-serving-cert: secret "maas-api-e2e-inf-a-453525-serving-cert" not found
Jul 23 19:37:36.660748 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:37:36.660706    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls podName:5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2 nodeName:}" failed. No retries permitted until 2026-07-23 19:37:37.16069095 +0000 UTC m=+2200.063277006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls") pod "maas-api-e2e-inf-a-453525-997864dd7-7wwg8" (UID: "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2") : secret "maas-api-e2e-inf-a-453525-serving-cert" not found
Jul 23 19:37:36.661093 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.661073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:36.668773 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:36.668751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwdc5\" (UniqueName: \"kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:37.165357 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:37.165320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:37.167942 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:37.167915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") pod \"maas-api-e2e-inf-a-453525-997864dd7-7wwg8\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:37.444866 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:37.444791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:37.575768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:37.575747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:37:37.577299 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:37:37.577275    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0129e6_61b2_4e75_aa71_e4ab22bd9bb2.slice/crio-81c3ca4ba59cbde78d5b1f3b1277dc13a89f9690256f4db67d0a077cbf7bbf87 WatchSource:0}: Error finding container 81c3ca4ba59cbde78d5b1f3b1277dc13a89f9690256f4db67d0a077cbf7bbf87: Status 404 returned error can't find the container with id 81c3ca4ba59cbde78d5b1f3b1277dc13a89f9690256f4db67d0a077cbf7bbf87
Jul 23 19:37:38.582664 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:38.582623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" event={"ID":"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2","Type":"ContainerStarted","Data":"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"}
Jul 23 19:37:38.583077 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:38.582676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" event={"ID":"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2","Type":"ContainerStarted","Data":"81c3ca4ba59cbde78d5b1f3b1277dc13a89f9690256f4db67d0a077cbf7bbf87"}
Jul 23 19:37:38.583077 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:38.582736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:38.618191 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:38.618098    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" podStartSLOduration=1.838117537 podStartE2EDuration="2.618082651s" totalImagesPullingTime="779.965114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:37:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:37:38.616055873 +0000 UTC m=+2201.518642016" watchObservedRunningTime="2026-07-23 19:37:38.618082651 +0000 UTC m=+2201.520668728"
Jul 23 19:37:41.231641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:41.231604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:37:44.425591 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.425559    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:37:44.429483 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.429457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.432016 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.431993    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d99fcd-serving-cert\""
Jul 23 19:37:44.440638 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.440611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:37:44.521734 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.521703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpw8p\" (UniqueName: \"kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.521921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.521745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.521921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.521851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.593085 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.593055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:37:44.622719 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.622690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpw8p\" (UniqueName: \"kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.622914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.622749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.622914 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.622861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.623413 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.623390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.625153 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.625135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.630326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.630304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpw8p\" (UniqueName: \"kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p\") pod \"maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.742788 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.742697    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:44.880925 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:44.880897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:37:44.882504 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:37:44.882481    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2ff28c_3207_411e_b86b_7a8956e40af1.slice/crio-02e7e3edb68c30d67d21aec6381767d6c896a3e07cf15d21867436d15b13a2b3 WatchSource:0}: Error finding container 02e7e3edb68c30d67d21aec6381767d6c896a3e07cf15d21867436d15b13a2b3: Status 404 returned error can't find the container with id 02e7e3edb68c30d67d21aec6381767d6c896a3e07cf15d21867436d15b13a2b3
Jul 23 19:37:45.619448 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:45.619405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" event={"ID":"de2ff28c-3207-411e-b86b-7a8956e40af1","Type":"ContainerStarted","Data":"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"}
Jul 23 19:37:45.619840 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:45.619470    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:45.619840 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:45.619491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" event={"ID":"de2ff28c-3207-411e-b86b-7a8956e40af1","Type":"ContainerStarted","Data":"02e7e3edb68c30d67d21aec6381767d6c896a3e07cf15d21867436d15b13a2b3"}
Jul 23 19:37:45.638964 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:45.638915    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" podStartSLOduration=1.306895697 podStartE2EDuration="1.638897415s" totalImagesPullingTime="332.001718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:37:45.636183785 +0000 UTC m=+2208.538769865" watchObservedRunningTime="2026-07-23 19:37:45.638897415 +0000 UTC m=+2208.541483513"
Jul 23 19:37:50.127978 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:50.127938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:37:51.631749 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:51.631716    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:37:57.753521 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:57.753490    2563 scope.go:117] "RemoveContainer" containerID="85e2de5c4e55128c85503a31eea1c45383d9542bc8e77434a3906eb900ab4866"
Jul 23 19:37:57.761776 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:57.761748    2563 scope.go:117] "RemoveContainer" containerID="7d6ce438b3b58decd5befa9b7d63e9b2c8207c840fffdd004284d9d79d78d450"
Jul 23 19:37:59.177590 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.177560    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:37:59.181985 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.181968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.184187 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.184164    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-453525\"/\"kube-root-ca.crt\""
Jul 23 19:37:59.185054 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.185035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-453525\"/\"test-model-453525-kserve-self-signed-certs\""
Jul 23 19:37:59.185054 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.185046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-453525\"/\"default-dockercfg-w8jdg\""
Jul 23 19:37:59.185199 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.185075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-453525\"/\"openshift-service-ca.crt\""
Jul 23 19:37:59.189868 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.189845    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:37:59.247006 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.246978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.247142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.247043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.247142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.247087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.247142 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.247117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.247257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.247147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.247257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.247162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hwwp\" (UniqueName: \"kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348293 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwwp\" (UniqueName: \"kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.348834 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.348779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.350645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.350624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.351037 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.351018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.355220 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.355202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwwp\" (UniqueName: \"kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp\") pod \"test-model-453525-kserve-57d5dc5bd5-vbg7j\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") " pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.492717 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.492658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:37:59.631859 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.631831    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:37:59.633630 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:37:59.633600    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7592a7c9_0f5c_4040_841b_c0eb8d97c183.slice/crio-9a0625e1d5caf0ca40951a61d18f8628a522809bea562a8b27f8567f8daafd1c WatchSource:0}: Error finding container 9a0625e1d5caf0ca40951a61d18f8628a522809bea562a8b27f8567f8daafd1c: Status 404 returned error can't find the container with id 9a0625e1d5caf0ca40951a61d18f8628a522809bea562a8b27f8567f8daafd1c
Jul 23 19:37:59.703205 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.703173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" event={"ID":"7592a7c9-0f5c-4040-841b-c0eb8d97c183","Type":"ContainerStarted","Data":"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"}
Jul 23 19:37:59.703318 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.703222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" event={"ID":"7592a7c9-0f5c-4040-841b-c0eb8d97c183","Type":"ContainerStarted","Data":"9a0625e1d5caf0ca40951a61d18f8628a522809bea562a8b27f8567f8daafd1c"}
Jul 23 19:37:59.723739 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:37:59.723697    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" podStartSLOduration=0.723682429 podStartE2EDuration="723.682429ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:37:59.723403608 +0000 UTC m=+2222.625989686" watchObservedRunningTime="2026-07-23 19:37:59.723682429 +0000 UTC m=+2222.626268485"
Jul 23 19:38:03.650110 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:03.650074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:38:09.493695 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:09.493649    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:38:09.494304 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:09.493710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:38:09.506347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:09.506325    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:38:09.774033 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:09.774001    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:38:18.669312 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.669276    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:38:18.674125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.674108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.678653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.677313    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d99fcd\"/\"default-dockercfg-rt5fv\""
Jul 23 19:38:18.678653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.677650    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d99fcd\"/\"openshift-service-ca.crt\""
Jul 23 19:38:18.678653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.677889    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d99fcd\"/\"kube-root-ca.crt\""
Jul 23 19:38:18.678653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.678095    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d99fcd\"/\"test-model-d99fcd-kserve-self-signed-certs\""
Jul 23 19:38:18.685215 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.684626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:38:18.810365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810324    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.810542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.810542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnsv\" (UniqueName: \"kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.810661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.810661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.810661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.810642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912082 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnsv\" (UniqueName: \"kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912581 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912638 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912638 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.912747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.912683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.914433 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.914410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.914698 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.914681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.921271 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.921222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnsv\" (UniqueName: \"kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv\") pod \"test-model-d99fcd-kserve-679c8f546c-x6zln\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") " pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:18.992326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:18.992293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:19.126934 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:19.126908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:38:19.129088 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:38:19.129051    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54de22a5_86a6_4117_ba82_14e7dbd3888a.slice/crio-2973ee32ffa30a6487a2bc3880cc78138b05f71c85405554be9575e7dea0b68a WatchSource:0}: Error finding container 2973ee32ffa30a6487a2bc3880cc78138b05f71c85405554be9575e7dea0b68a: Status 404 returned error can't find the container with id 2973ee32ffa30a6487a2bc3880cc78138b05f71c85405554be9575e7dea0b68a
Jul 23 19:38:19.816786 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:19.816740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" event={"ID":"54de22a5-86a6-4117-ba82-14e7dbd3888a","Type":"ContainerStarted","Data":"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"}
Jul 23 19:38:19.816786 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:19.816790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" event={"ID":"54de22a5-86a6-4117-ba82-14e7dbd3888a","Type":"ContainerStarted","Data":"2973ee32ffa30a6487a2bc3880cc78138b05f71c85405554be9575e7dea0b68a"}
Jul 23 19:38:19.834998 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:19.834954    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" podStartSLOduration=1.834940102 podStartE2EDuration="1.834940102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:38:19.833374208 +0000 UTC m=+2242.735960311" watchObservedRunningTime="2026-07-23 19:38:19.834940102 +0000 UTC m=+2242.737526180"
Jul 23 19:38:22.623583 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:22.623534    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:38:28.993109 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:28.993065    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:28.993568 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:28.993124    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:29.005858 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:29.005804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:29.879059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:29.879027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:38:36.934594 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:36.934552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:38:58.111105 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:58.111077    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:38:59.027610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:38:59.027558    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:39:02.926021 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:02.925993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:39:09.598924 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.598888    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:39:09.599395 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.599125    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" podUID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" containerName="maas-api" containerID="cri-o://c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f" gracePeriod=30
Jul 23 19:39:09.847435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.847411    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:39:09.944059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.943990    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca\") pod \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") "
Jul 23 19:39:09.944059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.944027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") pod \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") "
Jul 23 19:39:09.944059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.944053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwdc5\" (UniqueName: \"kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5\") pod \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\" (UID: \"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2\") "
Jul 23 19:39:09.944419 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.944401    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" (UID: "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:39:09.946085 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.946056    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5" (OuterVolumeSpecName: "kube-api-access-mwdc5") pod "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" (UID: "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2"). InnerVolumeSpecName "kube-api-access-mwdc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:09.946180 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:09.946106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" (UID: "5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:10.045197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.045165    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.045197 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.045194    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.045342 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.045207    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwdc5\" (UniqueName: \"kubernetes.io/projected/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2-kube-api-access-mwdc5\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.083896 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.083867    2563 generic.go:356] "Generic (PLEG): container finished" podID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" containerID="c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f" exitCode=0
Jul 23 19:39:10.084009 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.083913    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"
Jul 23 19:39:10.084009 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.083959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" event={"ID":"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2","Type":"ContainerDied","Data":"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"}
Jul 23 19:39:10.084130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.084051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8" event={"ID":"5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2","Type":"ContainerDied","Data":"81c3ca4ba59cbde78d5b1f3b1277dc13a89f9690256f4db67d0a077cbf7bbf87"}
Jul 23 19:39:10.084130 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.084085    2563 scope.go:117] "RemoveContainer" containerID="c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"
Jul 23 19:39:10.093539 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.093523    2563 scope.go:117] "RemoveContainer" containerID="c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"
Jul 23 19:39:10.093838 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:10.093800    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f\": container with ID starting with c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f not found: ID does not exist" containerID="c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"
Jul 23 19:39:10.093894 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.093848    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f"} err="failed to get container status \"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f\": rpc error: code = NotFound desc = could not find container \"c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f\": container with ID starting with c2bbc0a34f8e0d71afd726f9ff9284fb55f941cf0f638b3a0b005bdf082b8b8f not found: ID does not exist"
Jul 23 19:39:10.105038 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.105018    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:39:10.108932 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:10.108913    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-453525-997864dd7-7wwg8"]
Jul 23 19:39:11.522503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:11.522463    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" path="/var/lib/kubelet/pods/5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2/volumes"
Jul 23 19:39:13.134412 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.134382    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" podUID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" containerName="main" containerID="cri-o://3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1" gracePeriod=45
Jul 23 19:39:13.393608 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.393556    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:39:13.475373 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hwwp\" (UniqueName: \"kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475457    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475473    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475540    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache\") pod \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\" (UID: \"7592a7c9-0f5c-4040-841b-c0eb8d97c183\") "
Jul 23 19:39:13.475770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475593    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:13.475770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475768    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:13.476261 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475749    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home" (OuterVolumeSpecName: "home") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:13.476261 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.475839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache" (OuterVolumeSpecName: "model-cache") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:13.477844 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.477823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp" (OuterVolumeSpecName: "kube-api-access-2hwwp") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "kube-api-access-2hwwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:13.477923 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.477804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm" (OuterVolumeSpecName: "dshm") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:13.477923 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.477874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7592a7c9-0f5c-4040-841b-c0eb8d97c183" (UID: "7592a7c9-0f5c-4040-841b-c0eb8d97c183"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:13.576957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.576938    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:13.576957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.576956    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:13.577098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.576966    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hwwp\" (UniqueName: \"kubernetes.io/projected/7592a7c9-0f5c-4040-841b-c0eb8d97c183-kube-api-access-2hwwp\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:13.577098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.576976    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7592a7c9-0f5c-4040-841b-c0eb8d97c183-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:13.577098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:13.576985    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7592a7c9-0f5c-4040-841b-c0eb8d97c183-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.049611 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.049575    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" podUID="54de22a5-86a6-4117-ba82-14e7dbd3888a" containerName="main" containerID="cri-o://420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13" gracePeriod=45
Jul 23 19:39:14.106072 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.106048    2563 generic.go:356] "Generic (PLEG): container finished" podID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" containerID="3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1" exitCode=0
Jul 23 19:39:14.106200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.106103    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"
Jul 23 19:39:14.106200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.106126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" event={"ID":"7592a7c9-0f5c-4040-841b-c0eb8d97c183","Type":"ContainerDied","Data":"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"}
Jul 23 19:39:14.106200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.106166    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j" event={"ID":"7592a7c9-0f5c-4040-841b-c0eb8d97c183","Type":"ContainerDied","Data":"9a0625e1d5caf0ca40951a61d18f8628a522809bea562a8b27f8567f8daafd1c"}
Jul 23 19:39:14.106200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.106182    2563 scope.go:117] "RemoveContainer" containerID="3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"
Jul 23 19:39:14.122669 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.122644    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:39:14.125984 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.125965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-453525/test-model-453525-kserve-57d5dc5bd5-vbg7j"]
Jul 23 19:39:14.161336 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.161104    2563 scope.go:117] "RemoveContainer" containerID="3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"
Jul 23 19:39:14.161581 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:14.161453    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1\": container with ID starting with 3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1 not found: ID does not exist" containerID="3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"
Jul 23 19:39:14.161581 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.161486    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1"} err="failed to get container status \"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1\": rpc error: code = NotFound desc = could not find container \"3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1\": container with ID starting with 3ada6709fc2a368f957999e9c5a5ee6dbde5489bfa4ad6ecd7c56ca038e4a5c1 not found: ID does not exist"
Jul 23 19:39:14.304675 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.304617    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:39:14.383299 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmnsv\" (UniqueName: \"kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383299 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383300    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383586 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383319    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383586 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383586 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383586 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383453    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs\") pod \"54de22a5-86a6-4117-ba82-14e7dbd3888a\" (UID: \"54de22a5-86a6-4117-ba82-14e7dbd3888a\") "
Jul 23 19:39:14.383793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383644    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache" (OuterVolumeSpecName: "model-cache") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:14.383793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383666    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home" (OuterVolumeSpecName: "home") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:14.383793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383766    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:14.383982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383877    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.383982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383897    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.383982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.383912    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.385601 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.385580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:14.385684 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.385632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv" (OuterVolumeSpecName: "kube-api-access-rmnsv") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "kube-api-access-rmnsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:14.385748 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.385699    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm" (OuterVolumeSpecName: "dshm") pod "54de22a5-86a6-4117-ba82-14e7dbd3888a" (UID: "54de22a5-86a6-4117-ba82-14e7dbd3888a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:14.485239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.485203    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54de22a5-86a6-4117-ba82-14e7dbd3888a-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.485239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.485229    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmnsv\" (UniqueName: \"kubernetes.io/projected/54de22a5-86a6-4117-ba82-14e7dbd3888a-kube-api-access-rmnsv\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:14.485239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:14.485238    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54de22a5-86a6-4117-ba82-14e7dbd3888a-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:15.120298 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.120243    2563 generic.go:356] "Generic (PLEG): container finished" podID="54de22a5-86a6-4117-ba82-14e7dbd3888a" containerID="420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13" exitCode=0
Jul 23 19:39:15.120503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.120320    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"
Jul 23 19:39:15.120503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.120319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" event={"ID":"54de22a5-86a6-4117-ba82-14e7dbd3888a","Type":"ContainerDied","Data":"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"}
Jul 23 19:39:15.120503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.120466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln" event={"ID":"54de22a5-86a6-4117-ba82-14e7dbd3888a","Type":"ContainerDied","Data":"2973ee32ffa30a6487a2bc3880cc78138b05f71c85405554be9575e7dea0b68a"}
Jul 23 19:39:15.120503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.120494    2563 scope.go:117] "RemoveContainer" containerID="420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"
Jul 23 19:39:15.130662 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.130646    2563 scope.go:117] "RemoveContainer" containerID="420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"
Jul 23 19:39:15.130932 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:15.130914    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13\": container with ID starting with 420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13 not found: ID does not exist" containerID="420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"
Jul 23 19:39:15.130990 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.130941    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13"} err="failed to get container status \"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13\": rpc error: code = NotFound desc = could not find container \"420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13\": container with ID starting with 420a940405a1502e534650a35994b0cbcaedd85553fa826284ac66bb4d985e13 not found: ID does not exist"
Jul 23 19:39:15.142292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.142270    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:39:15.147777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.147757    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d99fcd/test-model-d99fcd-kserve-679c8f546c-x6zln"]
Jul 23 19:39:15.521345 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.521309    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54de22a5-86a6-4117-ba82-14e7dbd3888a" path="/var/lib/kubelet/pods/54de22a5-86a6-4117-ba82-14e7dbd3888a/volumes"
Jul 23 19:39:15.521890 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:15.521870    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" path="/var/lib/kubelet/pods/7592a7c9-0f5c-4040-841b-c0eb8d97c183/volumes"
Jul 23 19:39:19.233444 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:19.233405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:39:21.283009 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.282976    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"]
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283332    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" containerName="maas-api"
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283343    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" containerName="maas-api"
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283364    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" containerName="main"
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283370    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" containerName="main"
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283376    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54de22a5-86a6-4117-ba82-14e7dbd3888a" containerName="main"
Jul 23 19:39:21.283407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283382    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="54de22a5-86a6-4117-ba82-14e7dbd3888a" containerName="main"
Jul 23 19:39:21.283634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283435    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="54de22a5-86a6-4117-ba82-14e7dbd3888a" containerName="main"
Jul 23 19:39:21.283634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283446    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7592a7c9-0f5c-4040-841b-c0eb8d97c183" containerName="main"
Jul 23 19:39:21.283634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.283452    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b0129e6-61b2-4e75-aa71-e4ab22bd9bb2" containerName="maas-api"
Jul 23 19:39:21.287884 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.287863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:21.290385 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.290365    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:39:21.295645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.295622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"]
Jul 23 19:39:21.437952 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.437924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n\" (UID: \"1bc157e6-adc5-4a85-935a-37d24002c9ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:21.538977 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.538897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n\" (UID: \"1bc157e6-adc5-4a85-935a-37d24002c9ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:21.539504 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.539483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n\" (UID: \"1bc157e6-adc5-4a85-935a-37d24002c9ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:21.599979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.599952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:21.750938 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:21.750906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"]
Jul 23 19:39:21.752546 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:39:21.752517    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc157e6_adc5_4a85_935a_37d24002c9ba.slice/crio-860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830 WatchSource:0}: Error finding container 860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830: Status 404 returned error can't find the container with id 860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830
Jul 23 19:39:22.162661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:22.162628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n" event={"ID":"1bc157e6-adc5-4a85-935a-37d24002c9ba","Type":"ContainerStarted","Data":"378e768a74d4880296f105c92edfc043da73803388800598b163e12dd2c46a44"}
Jul 23 19:39:22.162793 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:22.162682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n" event={"ID":"1bc157e6-adc5-4a85-935a-37d24002c9ba","Type":"ContainerStarted","Data":"860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830"}
Jul 23 19:39:22.181684 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:22.181642    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n" podStartSLOduration=1.181628174 podStartE2EDuration="1.181628174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:39:22.180426875 +0000 UTC m=+2305.083012952" watchObservedRunningTime="2026-07-23 19:39:22.181628174 +0000 UTC m=+2305.084214251"
Jul 23 19:39:23.169115 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:23.169078    2563 generic.go:356] "Generic (PLEG): container finished" podID="1bc157e6-adc5-4a85-935a-37d24002c9ba" containerID="378e768a74d4880296f105c92edfc043da73803388800598b163e12dd2c46a44" exitCode=0
Jul 23 19:39:23.169499 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:23.169163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n" event={"ID":"1bc157e6-adc5-4a85-935a-37d24002c9ba","Type":"ContainerDied","Data":"378e768a74d4880296f105c92edfc043da73803388800598b163e12dd2c46a44"}
Jul 23 19:39:24.311313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:24.311286    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:24.462125 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:24.462041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca\") pod \"1bc157e6-adc5-4a85-935a-37d24002c9ba\" (UID: \"1bc157e6-adc5-4a85-935a-37d24002c9ba\") "
Jul 23 19:39:24.462407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:24.462388    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1bc157e6-adc5-4a85-935a-37d24002c9ba" (UID: "1bc157e6-adc5-4a85-935a-37d24002c9ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:39:24.563270 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:24.563235    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc157e6-adc5-4a85-935a-37d24002c9ba-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:25.181330 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:25.181302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"
Jul 23 19:39:25.181490 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:25.181303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n" event={"ID":"1bc157e6-adc5-4a85-935a-37d24002c9ba","Type":"ContainerDied","Data":"860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830"}
Jul 23 19:39:25.181490 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:25.181423    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="860dd851e1c6e9594fb92f0b9faab2d00c005fec3f8520ff553d4c253178e830"
Jul 23 19:39:31.666609 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:31.666575    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:39:31.667140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:31.666787    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" podUID="de2ff28c-3207-411e-b86b-7a8956e40af1" containerName="maas-api" containerID="cri-o://27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6" gracePeriod=30
Jul 23 19:39:31.913119 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:31.913091    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:39:32.023178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.023147    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpw8p\" (UniqueName: \"kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p\") pod \"de2ff28c-3207-411e-b86b-7a8956e40af1\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") "
Jul 23 19:39:32.023341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.023210    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls\") pod \"de2ff28c-3207-411e-b86b-7a8956e40af1\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") "
Jul 23 19:39:32.023341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.023283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca\") pod \"de2ff28c-3207-411e-b86b-7a8956e40af1\" (UID: \"de2ff28c-3207-411e-b86b-7a8956e40af1\") "
Jul 23 19:39:32.023676 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.023645    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de2ff28c-3207-411e-b86b-7a8956e40af1" (UID: "de2ff28c-3207-411e-b86b-7a8956e40af1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:39:32.025340 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.025317    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de2ff28c-3207-411e-b86b-7a8956e40af1" (UID: "de2ff28c-3207-411e-b86b-7a8956e40af1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:32.025436 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.025358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p" (OuterVolumeSpecName: "kube-api-access-hpw8p") pod "de2ff28c-3207-411e-b86b-7a8956e40af1" (UID: "de2ff28c-3207-411e-b86b-7a8956e40af1"). InnerVolumeSpecName "kube-api-access-hpw8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:32.123900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.123874    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpw8p\" (UniqueName: \"kubernetes.io/projected/de2ff28c-3207-411e-b86b-7a8956e40af1-kube-api-access-hpw8p\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:32.123900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.123898    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2ff28c-3207-411e-b86b-7a8956e40af1-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:32.124037 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.123907    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2ff28c-3207-411e-b86b-7a8956e40af1-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:39:32.220226 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.220195    2563 generic.go:356] "Generic (PLEG): container finished" podID="de2ff28c-3207-411e-b86b-7a8956e40af1" containerID="27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6" exitCode=0
Jul 23 19:39:32.220340 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.220243    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"
Jul 23 19:39:32.220340 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.220288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" event={"ID":"de2ff28c-3207-411e-b86b-7a8956e40af1","Type":"ContainerDied","Data":"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"}
Jul 23 19:39:32.220431 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.220372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh" event={"ID":"de2ff28c-3207-411e-b86b-7a8956e40af1","Type":"ContainerDied","Data":"02e7e3edb68c30d67d21aec6381767d6c896a3e07cf15d21867436d15b13a2b3"}
Jul 23 19:39:32.220431 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.220397    2563 scope.go:117] "RemoveContainer" containerID="27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"
Jul 23 19:39:32.229480 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.229462    2563 scope.go:117] "RemoveContainer" containerID="27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"
Jul 23 19:39:32.229731 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:32.229713    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6\": container with ID starting with 27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6 not found: ID does not exist" containerID="27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"
Jul 23 19:39:32.229784 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.229739    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6"} err="failed to get container status \"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6\": rpc error: code = NotFound desc = could not find container \"27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6\": container with ID starting with 27333a4c42f6468f4375f823fb9979c892f2b35d70fff7e691c2ebb1a7c1f1b6 not found: ID does not exist"
Jul 23 19:39:32.241434 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.241413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:39:32.245288 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:32.245268    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d99fcd-79f77b6d77-vdlnh"]
Jul 23 19:39:33.522009 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:33.521979    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de2ff28c-3207-411e-b86b-7a8956e40af1" path="/var/lib/kubelet/pods/de2ff28c-3207-411e-b86b-7a8956e40af1/volumes"
Jul 23 19:39:34.539322 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:34.539279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:39:36.346684 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:36.346655    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"]
Jul 23 19:39:36.349249 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:36.349231    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d99fcd-2r86n"]
Jul 23 19:39:37.522181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:37.522147    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc157e6-adc5-4a85-935a-37d24002c9ba" path="/var/lib/kubelet/pods/1bc157e6-adc5-4a85-935a-37d24002c9ba/volumes"
Jul 23 19:39:47.040723 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.040683    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:39:47.041221 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041203    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2ff28c-3207-411e-b86b-7a8956e40af1" containerName="maas-api"
Jul 23 19:39:47.041221 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041222    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2ff28c-3207-411e-b86b-7a8956e40af1" containerName="maas-api"
Jul 23 19:39:47.041348 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041258    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc157e6-adc5-4a85-935a-37d24002c9ba" containerName="revoke-keys"
Jul 23 19:39:47.041348 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041268    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc157e6-adc5-4a85-935a-37d24002c9ba" containerName="revoke-keys"
Jul 23 19:39:47.041446 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041356    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bc157e6-adc5-4a85-935a-37d24002c9ba" containerName="revoke-keys"
Jul 23 19:39:47.041446 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.041368    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2ff28c-3207-411e-b86b-7a8956e40af1" containerName="maas-api"
Jul 23 19:39:47.044639 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.044618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.048544 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.048524    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:39:47.048707 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.048686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-bd74ec-serving-cert\""
Jul 23 19:39:47.048790 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.048736    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:39:47.049344 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.049329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:39:47.054964 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.054945    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:39:47.135107 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.135073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vzm7\" (UniqueName: \"kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.135260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.135140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.135260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.135162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.235726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.235689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.235726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.235729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.236010 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.235804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vzm7\" (UniqueName: \"kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.236010 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:47.235866    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-serving-cert: secret "maas-api-e2e-api-a-bd74ec-serving-cert" not found
Jul 23 19:39:47.236010 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:47.235947    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls podName:63407e58-2804-42c9-831c-1f615c5dac0a nodeName:}" failed. No retries permitted until 2026-07-23 19:39:47.735926461 +0000 UTC m=+2330.638512535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls") pod "maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" (UID: "63407e58-2804-42c9-831c-1f615c5dac0a") : secret "maas-api-e2e-api-a-bd74ec-serving-cert" not found
Jul 23 19:39:47.236562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.236537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.245045 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.245016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vzm7\" (UniqueName: \"kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.738887 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.738852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.741231 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.741213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") pod \"maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:47.956227 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:47.956185    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:48.098031 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:48.098007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:39:48.099532 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:39:48.099502    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63407e58_2804_42c9_831c_1f615c5dac0a.slice/crio-b0fa1bbdf588660e52ebc7aa83d974c6b8e9eb7c781b86679f8f1cf962b5fd99 WatchSource:0}: Error finding container b0fa1bbdf588660e52ebc7aa83d974c6b8e9eb7c781b86679f8f1cf962b5fd99: Status 404 returned error can't find the container with id b0fa1bbdf588660e52ebc7aa83d974c6b8e9eb7c781b86679f8f1cf962b5fd99
Jul 23 19:39:48.310855 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:48.310753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" event={"ID":"63407e58-2804-42c9-831c-1f615c5dac0a","Type":"ContainerStarted","Data":"b0fa1bbdf588660e52ebc7aa83d974c6b8e9eb7c781b86679f8f1cf962b5fd99"}
Jul 23 19:39:49.317068 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:49.317033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" event={"ID":"63407e58-2804-42c9-831c-1f615c5dac0a","Type":"ContainerStarted","Data":"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"}
Jul 23 19:39:49.317455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:49.317163    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:49.341089 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:49.341042    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" podStartSLOduration=1.882123432 podStartE2EDuration="2.341028006s" totalImagesPullingTime="458.904574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:39:49.339175261 +0000 UTC m=+2332.241761339" watchObservedRunningTime="2026-07-23 19:39:49.341028006 +0000 UTC m=+2332.243614081"
Jul 23 19:39:52.343003 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:52.342925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:39:55.074574 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.074530    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:39:55.078426 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.078407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.081395 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.081374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7e6d6b-serving-cert\""
Jul 23 19:39:55.091789 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.091766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:39:55.106699 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.106671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv2pz\" (UniqueName: \"kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.106806 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.106720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.106806 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.106750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.208069 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.208030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv2pz\" (UniqueName: \"kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.208254 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.208088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.208254 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.208113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.208384 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:55.208253    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-serving-cert: secret "maas-api-e2e-api-b-7e6d6b-serving-cert" not found
Jul 23 19:39:55.208384 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:39:55.208340    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls podName:dcf016ca-33de-43ed-92e0-6dec0654ce7b nodeName:}" failed. No retries permitted until 2026-07-23 19:39:55.708317917 +0000 UTC m=+2338.610903976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls") pod "maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" (UID: "dcf016ca-33de-43ed-92e0-6dec0654ce7b") : secret "maas-api-e2e-api-b-7e6d6b-serving-cert" not found
Jul 23 19:39:55.208893 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.208872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.219415 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.219385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv2pz\" (UniqueName: \"kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.328853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.328752    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:39:55.714119 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.714040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.716376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.716348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") pod \"maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:55.991161 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:55.991075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:56.129249 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:56.129218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:39:56.130500 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:39:56.130474    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf016ca_33de_43ed_92e0_6dec0654ce7b.slice/crio-4d30e329199c9d4fd0a58c589a12345def19f03e55c8b70577c271af299473c2 WatchSource:0}: Error finding container 4d30e329199c9d4fd0a58c589a12345def19f03e55c8b70577c271af299473c2: Status 404 returned error can't find the container with id 4d30e329199c9d4fd0a58c589a12345def19f03e55c8b70577c271af299473c2
Jul 23 19:39:56.131912 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:56.131891    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:39:56.354595 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:56.354565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" event={"ID":"dcf016ca-33de-43ed-92e0-6dec0654ce7b","Type":"ContainerStarted","Data":"4d30e329199c9d4fd0a58c589a12345def19f03e55c8b70577c271af299473c2"}
Jul 23 19:39:57.364297 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:57.364257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" event={"ID":"dcf016ca-33de-43ed-92e0-6dec0654ce7b","Type":"ContainerStarted","Data":"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"}
Jul 23 19:39:57.364696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:57.364315    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:39:57.383558 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:39:57.383507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" podStartSLOduration=1.928918911 podStartE2EDuration="2.383493047s" totalImagesPullingTime="454.574136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:39:57.381677657 +0000 UTC m=+2340.284263734" watchObservedRunningTime="2026-07-23 19:39:57.383493047 +0000 UTC m=+2340.286079125"
Jul 23 19:40:03.375763 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:03.375736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:40:05.224780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:05.224744    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:40:07.467322 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.467286    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"]
Jul 23 19:40:07.470986 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.470964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:07.472979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.472958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:40:07.480953 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.480928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"]
Jul 23 19:40:07.618699 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.618664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8\" (UID: \"9fd8c4cf-6904-41a7-a549-f76b99695cdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:07.719951 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.719869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8\" (UID: \"9fd8c4cf-6904-41a7-a549-f76b99695cdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:07.720470 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.720452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8\" (UID: \"9fd8c4cf-6904-41a7-a549-f76b99695cdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:07.781760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.781737    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:07.937690 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:07.933804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"]
Jul 23 19:40:08.422213 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:08.422177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8" event={"ID":"9fd8c4cf-6904-41a7-a549-f76b99695cdb","Type":"ContainerStarted","Data":"4f67a3663e7a9233a561c4bc560c2b843947ca8f418ae40a3db50c58a6d66d8a"}
Jul 23 19:40:08.422391 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:08.422233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8" event={"ID":"9fd8c4cf-6904-41a7-a549-f76b99695cdb","Type":"ContainerStarted","Data":"3a61da6fe148eeb09e850df02a6559e75661bbbc560b1a94a151a88240a25980"}
Jul 23 19:40:08.438305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:08.438263    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8" podStartSLOduration=1.43824828 podStartE2EDuration="1.43824828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:40:08.436223892 +0000 UTC m=+2351.338809998" watchObservedRunningTime="2026-07-23 19:40:08.43824828 +0000 UTC m=+2351.340834357"
Jul 23 19:40:09.428864 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:09.428828    2563 generic.go:356] "Generic (PLEG): container finished" podID="9fd8c4cf-6904-41a7-a549-f76b99695cdb" containerID="4f67a3663e7a9233a561c4bc560c2b843947ca8f418ae40a3db50c58a6d66d8a" exitCode=0
Jul 23 19:40:09.429226 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:09.428913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8" event={"ID":"9fd8c4cf-6904-41a7-a549-f76b99695cdb","Type":"ContainerDied","Data":"4f67a3663e7a9233a561c4bc560c2b843947ca8f418ae40a3db50c58a6d66d8a"}
Jul 23 19:40:10.566537 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:10.566513    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:10.645562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:10.645532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca\") pod \"9fd8c4cf-6904-41a7-a549-f76b99695cdb\" (UID: \"9fd8c4cf-6904-41a7-a549-f76b99695cdb\") "
Jul 23 19:40:10.645872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:10.645851    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fd8c4cf-6904-41a7-a549-f76b99695cdb" (UID: "9fd8c4cf-6904-41a7-a549-f76b99695cdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:40:10.746534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:10.746477    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fd8c4cf-6904-41a7-a549-f76b99695cdb-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:11.440018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:11.439991    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"
Jul 23 19:40:11.440018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:11.440008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8" event={"ID":"9fd8c4cf-6904-41a7-a549-f76b99695cdb","Type":"ContainerDied","Data":"3a61da6fe148eeb09e850df02a6559e75661bbbc560b1a94a151a88240a25980"}
Jul 23 19:40:11.440224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:11.440058    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a61da6fe148eeb09e850df02a6559e75661bbbc560b1a94a151a88240a25980"
Jul 23 19:40:22.515464 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.515431    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:40:22.515945 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.515896    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" podUID="63407e58-2804-42c9-831c-1f615c5dac0a" containerName="maas-api" containerID="cri-o://5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac" gracePeriod=30
Jul 23 19:40:22.766779 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.766718    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:40:22.841008 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.840978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") pod \"63407e58-2804-42c9-831c-1f615c5dac0a\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") "
Jul 23 19:40:22.841161 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.841075    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vzm7\" (UniqueName: \"kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7\") pod \"63407e58-2804-42c9-831c-1f615c5dac0a\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") "
Jul 23 19:40:22.841161 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.841137    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca\") pod \"63407e58-2804-42c9-831c-1f615c5dac0a\" (UID: \"63407e58-2804-42c9-831c-1f615c5dac0a\") "
Jul 23 19:40:22.841458 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.841434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63407e58-2804-42c9-831c-1f615c5dac0a" (UID: "63407e58-2804-42c9-831c-1f615c5dac0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:40:22.843027 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.843003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63407e58-2804-42c9-831c-1f615c5dac0a" (UID: "63407e58-2804-42c9-831c-1f615c5dac0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:40:22.843148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.843091    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7" (OuterVolumeSpecName: "kube-api-access-2vzm7") pod "63407e58-2804-42c9-831c-1f615c5dac0a" (UID: "63407e58-2804-42c9-831c-1f615c5dac0a"). InnerVolumeSpecName "kube-api-access-2vzm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:40:22.942213 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.942188    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63407e58-2804-42c9-831c-1f615c5dac0a-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:22.942213 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.942209    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63407e58-2804-42c9-831c-1f615c5dac0a-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:22.942361 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:22.942218    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vzm7\" (UniqueName: \"kubernetes.io/projected/63407e58-2804-42c9-831c-1f615c5dac0a-kube-api-access-2vzm7\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:23.509778 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.509742    2563 generic.go:356] "Generic (PLEG): container finished" podID="63407e58-2804-42c9-831c-1f615c5dac0a" containerID="5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac" exitCode=0
Jul 23 19:40:23.509971 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.509792    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"
Jul 23 19:40:23.509971 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.509804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" event={"ID":"63407e58-2804-42c9-831c-1f615c5dac0a","Type":"ContainerDied","Data":"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"}
Jul 23 19:40:23.509971 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.509880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj" event={"ID":"63407e58-2804-42c9-831c-1f615c5dac0a","Type":"ContainerDied","Data":"b0fa1bbdf588660e52ebc7aa83d974c6b8e9eb7c781b86679f8f1cf962b5fd99"}
Jul 23 19:40:23.509971 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.509910    2563 scope.go:117] "RemoveContainer" containerID="5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"
Jul 23 19:40:23.520166 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.519971    2563 scope.go:117] "RemoveContainer" containerID="5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"
Jul 23 19:40:23.520381 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:40:23.520292    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac\": container with ID starting with 5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac not found: ID does not exist" containerID="5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"
Jul 23 19:40:23.520381 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.520314    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac"} err="failed to get container status \"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac\": rpc error: code = NotFound desc = could not find container \"5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac\": container with ID starting with 5029c01067eda2bf50eb4d1684578985f637eb14540b300469c6c313af67f2ac not found: ID does not exist"
Jul 23 19:40:23.534412 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.534386    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:40:23.539847 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:23.539799    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bd74ec-55fd6bf8df-5szcj"]
Jul 23 19:40:24.927790 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:24.927757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:40:25.528773 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:25.528741    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63407e58-2804-42c9-831c-1f615c5dac0a" path="/var/lib/kubelet/pods/63407e58-2804-42c9-831c-1f615c5dac0a/volumes"
Jul 23 19:40:27.534437 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:27.534406    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"]
Jul 23 19:40:27.536633 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:27.536607    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bd74ec-mjlb8"]
Jul 23 19:40:29.521520 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:29.521491    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fd8c4cf-6904-41a7-a549-f76b99695cdb" path="/var/lib/kubelet/pods/9fd8c4cf-6904-41a7-a549-f76b99695cdb/volumes"
Jul 23 19:40:34.558882 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.558850    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"]
Jul 23 19:40:34.559484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559380    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63407e58-2804-42c9-831c-1f615c5dac0a" containerName="maas-api"
Jul 23 19:40:34.559484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559399    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63407e58-2804-42c9-831c-1f615c5dac0a" containerName="maas-api"
Jul 23 19:40:34.559484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559423    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd8c4cf-6904-41a7-a549-f76b99695cdb" containerName="revoke-keys"
Jul 23 19:40:34.559484 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559432    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd8c4cf-6904-41a7-a549-f76b99695cdb" containerName="revoke-keys"
Jul 23 19:40:34.559725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559517    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd8c4cf-6904-41a7-a549-f76b99695cdb" containerName="revoke-keys"
Jul 23 19:40:34.559725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.559529    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63407e58-2804-42c9-831c-1f615c5dac0a" containerName="maas-api"
Jul 23 19:40:34.562643 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.562619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:34.564853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.564835    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:40:34.573063 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.572997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"]
Jul 23 19:40:34.737498 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.737463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5\" (UID: \"20fbac9b-048a-4849-810c-1d6b5316b36e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:34.838428 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.838351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5\" (UID: \"20fbac9b-048a-4849-810c-1d6b5316b36e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:34.839048 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.839026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5\" (UID: \"20fbac9b-048a-4849-810c-1d6b5316b36e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:34.874019 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:34.873988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:35.003925 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:35.003896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"]
Jul 23 19:40:35.005474 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:40:35.005446    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fbac9b_048a_4849_810c_1d6b5316b36e.slice/crio-84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323 WatchSource:0}: Error finding container 84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323: Status 404 returned error can't find the container with id 84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323
Jul 23 19:40:35.583681 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:35.583645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5" event={"ID":"20fbac9b-048a-4849-810c-1d6b5316b36e","Type":"ContainerStarted","Data":"61d560c5b7bf504acee5ec476437f968a81de3ac5af32d6a34a30cbc2fe20c37"}
Jul 23 19:40:35.584081 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:35.583697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5" event={"ID":"20fbac9b-048a-4849-810c-1d6b5316b36e","Type":"ContainerStarted","Data":"84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323"}
Jul 23 19:40:35.600332 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:35.600278    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5" podStartSLOduration=1.6002619120000001 podStartE2EDuration="1.600261912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:40:35.598427935 +0000 UTC m=+2378.501014013" watchObservedRunningTime="2026-07-23 19:40:35.600261912 +0000 UTC m=+2378.502847990"
Jul 23 19:40:36.589799 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:36.589765    2563 generic.go:356] "Generic (PLEG): container finished" podID="20fbac9b-048a-4849-810c-1d6b5316b36e" containerID="61d560c5b7bf504acee5ec476437f968a81de3ac5af32d6a34a30cbc2fe20c37" exitCode=0
Jul 23 19:40:36.590212 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:36.589854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5" event={"ID":"20fbac9b-048a-4849-810c-1d6b5316b36e","Type":"ContainerDied","Data":"61d560c5b7bf504acee5ec476437f968a81de3ac5af32d6a34a30cbc2fe20c37"}
Jul 23 19:40:37.725612 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:37.725590    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:37.761842 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:37.761792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca\") pod \"20fbac9b-048a-4849-810c-1d6b5316b36e\" (UID: \"20fbac9b-048a-4849-810c-1d6b5316b36e\") "
Jul 23 19:40:37.762137 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:37.762116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20fbac9b-048a-4849-810c-1d6b5316b36e" (UID: "20fbac9b-048a-4849-810c-1d6b5316b36e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:40:37.863007 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:37.862950    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20fbac9b-048a-4849-810c-1d6b5316b36e-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:38.602013 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:38.601985    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"
Jul 23 19:40:38.602178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:38.602031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5" event={"ID":"20fbac9b-048a-4849-810c-1d6b5316b36e","Type":"ContainerDied","Data":"84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323"}
Jul 23 19:40:38.602178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:38.602077    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84305e405bad468dc023e614ca447a0d12465dd98d0bbdff6521c81eb23c4323"
Jul 23 19:40:49.932527 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:49.932496    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:40:49.932928 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:49.932768    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" podUID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" containerName="maas-api" containerID="cri-o://741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55" gracePeriod=30
Jul 23 19:40:50.186682 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.186625    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:40:50.257774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.257748    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca\") pod \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") "
Jul 23 19:40:50.257905 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.257796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") pod \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") "
Jul 23 19:40:50.257905 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.257843    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv2pz\" (UniqueName: \"kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz\") pod \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\" (UID: \"dcf016ca-33de-43ed-92e0-6dec0654ce7b\") "
Jul 23 19:40:50.258083 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.258062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcf016ca-33de-43ed-92e0-6dec0654ce7b" (UID: "dcf016ca-33de-43ed-92e0-6dec0654ce7b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:40:50.259768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.259741    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcf016ca-33de-43ed-92e0-6dec0654ce7b" (UID: "dcf016ca-33de-43ed-92e0-6dec0654ce7b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:40:50.259979 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.259900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz" (OuterVolumeSpecName: "kube-api-access-cv2pz") pod "dcf016ca-33de-43ed-92e0-6dec0654ce7b" (UID: "dcf016ca-33de-43ed-92e0-6dec0654ce7b"). InnerVolumeSpecName "kube-api-access-cv2pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:40:50.358829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.358779    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf016ca-33de-43ed-92e0-6dec0654ce7b-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:50.358829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.358832    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf016ca-33de-43ed-92e0-6dec0654ce7b-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:50.358829 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.358843    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv2pz\" (UniqueName: \"kubernetes.io/projected/dcf016ca-33de-43ed-92e0-6dec0654ce7b-kube-api-access-cv2pz\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:40:50.664560 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.664527    2563 generic.go:356] "Generic (PLEG): container finished" podID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" containerID="741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55" exitCode=0
Jul 23 19:40:50.664725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.664565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"
Jul 23 19:40:50.664725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.664609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" event={"ID":"dcf016ca-33de-43ed-92e0-6dec0654ce7b","Type":"ContainerDied","Data":"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"}
Jul 23 19:40:50.664725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.664658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb" event={"ID":"dcf016ca-33de-43ed-92e0-6dec0654ce7b","Type":"ContainerDied","Data":"4d30e329199c9d4fd0a58c589a12345def19f03e55c8b70577c271af299473c2"}
Jul 23 19:40:50.664725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.664681    2563 scope.go:117] "RemoveContainer" containerID="741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"
Jul 23 19:40:50.680850 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.680832    2563 scope.go:117] "RemoveContainer" containerID="741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"
Jul 23 19:40:50.681097 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:40:50.681078    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55\": container with ID starting with 741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55 not found: ID does not exist" containerID="741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"
Jul 23 19:40:50.681163 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.681103    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55"} err="failed to get container status \"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55\": rpc error: code = NotFound desc = could not find container \"741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55\": container with ID starting with 741fcdba0caa45e0e9d9b4b1efeab1cf52be2e3d2d62e3f9e39643231a406a55 not found: ID does not exist"
Jul 23 19:40:50.688360 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.688336    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:40:50.692368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:50.692348    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7e6d6b-6479bc59c-9bqmb"]
Jul 23 19:40:51.522266 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:51.522230    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" path="/var/lib/kubelet/pods/dcf016ca-33de-43ed-92e0-6dec0654ce7b/volumes"
Jul 23 19:40:54.627612 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:54.627566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"]
Jul 23 19:40:54.631252 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:54.631226    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7e6d6b-l2kx5"]
Jul 23 19:40:54.820323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:54.820289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:40:55.521781 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:55.521753    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fbac9b-048a-4849-810c-1d6b5316b36e" path="/var/lib/kubelet/pods/20fbac9b-048a-4849-810c-1d6b5316b36e/volumes"
Jul 23 19:40:57.817884 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:40:57.817857    2563 scope.go:117] "RemoveContainer" containerID="ed50a907928b59310c986860b3e7514062a95a16234552291d236c999f7aee9e"
Jul 23 19:41:05.641619 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.641583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:41:05.642123 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.641994    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fbac9b-048a-4849-810c-1d6b5316b36e" containerName="revoke-keys"
Jul 23 19:41:05.642123 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.642012    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fbac9b-048a-4849-810c-1d6b5316b36e" containerName="revoke-keys"
Jul 23 19:41:05.642123 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.642028    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" containerName="maas-api"
Jul 23 19:41:05.642123 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.642037    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" containerName="maas-api"
Jul 23 19:41:05.642303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.642161    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcf016ca-33de-43ed-92e0-6dec0654ce7b" containerName="maas-api"
Jul 23 19:41:05.642303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.642173    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fbac9b-048a-4849-810c-1d6b5316b36e" containerName="revoke-keys"
Jul 23 19:41:05.644978 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.644963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.647720 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.647698    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d5ee7b-serving-cert\""
Jul 23 19:41:05.647856 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.647747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jn4k8\""
Jul 23 19:41:05.648462 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.648446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:41:05.649033 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.649015    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:41:05.664634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.664608    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:41:05.681345 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.681322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8d6h\" (UniqueName: \"kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.681472 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.681395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.681472 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.681420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.781758 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.781728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.781922 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.781761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.781922 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.781838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8d6h\" (UniqueName: \"kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.781922 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:41:05.781891    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-serving-cert: secret "maas-api-e2e-shared-a-d5ee7b-serving-cert" not found
Jul 23 19:41:05.782084 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:41:05.781953    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls podName:014fd71e-17be-403f-8736-b2027a46435c nodeName:}" failed. No retries permitted until 2026-07-23 19:41:06.281936084 +0000 UTC m=+2409.184522139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls") pod "maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" (UID: "014fd71e-17be-403f-8736-b2027a46435c") : secret "maas-api-e2e-shared-a-d5ee7b-serving-cert" not found
Jul 23 19:41:05.782405 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.782386    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:05.793084 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:05.793060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8d6h\" (UniqueName: \"kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:06.286322 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:06.286286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:06.288625 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:06.288603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") pod \"maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:06.556313 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:06.556242    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:06.698372 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:06.698347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:41:06.699496 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:41:06.699471    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014fd71e_17be_403f_8736_b2027a46435c.slice/crio-2bc6a0308da4cd72c3efcf1cf7fdcbe7224226377cd175100791df256a8089aa WatchSource:0}: Error finding container 2bc6a0308da4cd72c3efcf1cf7fdcbe7224226377cd175100791df256a8089aa: Status 404 returned error can't find the container with id 2bc6a0308da4cd72c3efcf1cf7fdcbe7224226377cd175100791df256a8089aa
Jul 23 19:41:06.757715 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:06.757680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" event={"ID":"014fd71e-17be-403f-8736-b2027a46435c","Type":"ContainerStarted","Data":"2bc6a0308da4cd72c3efcf1cf7fdcbe7224226377cd175100791df256a8089aa"}
Jul 23 19:41:07.764177 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:07.764136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" event={"ID":"014fd71e-17be-403f-8736-b2027a46435c","Type":"ContainerStarted","Data":"6648590038e7d2ceb440c6c51d6eef42f0c50515d6134eeb4c4e495df0352b65"}
Jul 23 19:41:07.764555 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:07.764233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:07.790912 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:07.790866    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" podStartSLOduration=2.441862485 podStartE2EDuration="2.790853714s" totalImagesPullingTime="348.991229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:07.788231552 +0000 UTC m=+2410.690817629" watchObservedRunningTime="2026-07-23 19:41:07.790853714 +0000 UTC m=+2410.693439794"
Jul 23 19:41:10.924594 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:10.924562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:41:13.530680 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.530639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:41:13.534471 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.534447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.536733 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.536707    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4a426a-serving-cert\""
Jul 23 19:41:13.545503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.545481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:41:13.644012 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.643984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75sw\" (UniqueName: \"kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.644175 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.644024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.644175 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.644123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.745554 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.745530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75sw\" (UniqueName: \"kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.745696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.745560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.745696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.745595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.746285 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.746265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.747800 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.747781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.755257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.755236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75sw\" (UniqueName: \"kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw\") pod \"maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.773991 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.773966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:41:13.845722 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.845653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:13.982588 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:13.982557    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:41:13.984637 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:41:13.984607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87dff83d_92cd_440b_8f31_50ebf01386ac.slice/crio-0b29891923c371d1aa3060dbdc0e57805a3bbb2c40cc804f715ff4a1f6145d75 WatchSource:0}: Error finding container 0b29891923c371d1aa3060dbdc0e57805a3bbb2c40cc804f715ff4a1f6145d75: Status 404 returned error can't find the container with id 0b29891923c371d1aa3060dbdc0e57805a3bbb2c40cc804f715ff4a1f6145d75
Jul 23 19:41:14.803580 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:14.803536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" event={"ID":"87dff83d-92cd-440b-8f31-50ebf01386ac","Type":"ContainerStarted","Data":"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"}
Jul 23 19:41:14.804034 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:14.803646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" event={"ID":"87dff83d-92cd-440b-8f31-50ebf01386ac","Type":"ContainerStarted","Data":"0b29891923c371d1aa3060dbdc0e57805a3bbb2c40cc804f715ff4a1f6145d75"}
Jul 23 19:41:14.804034 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:14.803781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:14.821124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:14.821080    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" podStartSLOduration=1.438346726 podStartE2EDuration="1.821066047s" totalImagesPullingTime="382.719321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:14.819478995 +0000 UTC m=+2417.722065088" watchObservedRunningTime="2026-07-23 19:41:14.821066047 +0000 UTC m=+2417.723652123"
Jul 23 19:41:20.815999 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:20.815931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:41:22.919245 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:22.919215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:41:36.568187 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.568153    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:41:36.571842 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.571800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.575029 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.575006    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"default-dockercfg-bgfz6\""
Jul 23 19:41:36.575029 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.575025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"kube-root-ca.crt\""
Jul 23 19:41:36.575193 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.575025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"auth-test-model-d5ee7b-kserve-self-signed-certs\""
Jul 23 19:41:36.575193 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.575028    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"openshift-service-ca.crt\""
Jul 23 19:41:36.581325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.581302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:41:36.733439 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.733600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.733600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.733600 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.733741 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.733741 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.733637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c94q9\" (UniqueName: \"kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.834870 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.834870 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.834870 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c94q9\" (UniqueName: \"kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835094 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835094 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835094 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.834934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835268 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.835243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835332 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.835313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.835376 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.835348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.837082 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.837063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.837360 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.837341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.842964 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.842944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c94q9\" (UniqueName: \"kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9\") pod \"auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") " pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:36.882309 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:36.882286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:37.013780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:37.013754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:41:37.015533 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:41:37.015504    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ee7abd2_727f_430f_9490_7fe439b0597b.slice/crio-c664d76c6d9d6cb31e7bf750abe956b201f91332233f3b0c1a6e161eb774596d WatchSource:0}: Error finding container c664d76c6d9d6cb31e7bf750abe956b201f91332233f3b0c1a6e161eb774596d: Status 404 returned error can't find the container with id c664d76c6d9d6cb31e7bf750abe956b201f91332233f3b0c1a6e161eb774596d
Jul 23 19:41:37.927689 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:37.927651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" event={"ID":"3ee7abd2-727f-430f-9490-7fe439b0597b","Type":"ContainerStarted","Data":"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"}
Jul 23 19:41:37.928091 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:37.927700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" event={"ID":"3ee7abd2-727f-430f-9490-7fe439b0597b","Type":"ContainerStarted","Data":"c664d76c6d9d6cb31e7bf750abe956b201f91332233f3b0c1a6e161eb774596d"}
Jul 23 19:41:37.945428 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:37.945382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" podStartSLOduration=1.945368669 podStartE2EDuration="1.945368669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:37.943946342 +0000 UTC m=+2440.846532432" watchObservedRunningTime="2026-07-23 19:41:37.945368669 +0000 UTC m=+2440.847954746"
Jul 23 19:41:40.117316 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:40.117282    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:41:46.882926 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:46.882889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:46.883303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:46.882951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:46.896038 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:46.896008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:46.985555 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:46.985529    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:41:52.465422 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.465384    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:41:52.470131 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.470109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.472474 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.472454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"kube-root-ca.crt\""
Jul 23 19:41:52.473064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.473040    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"openshift-service-ca.crt\""
Jul 23 19:41:52.473227 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.473207    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"default-dockercfg-lqsjj\""
Jul 23 19:41:52.473326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.473312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"auth-test-model-4a426a-kserve-self-signed-certs\""
Jul 23 19:41:52.479760 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.479740    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:41:52.568026 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.567996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.568173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.568031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.568173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.568068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9p96\" (UniqueName: \"kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.568173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.568113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.568173 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.568136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.568352 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.568196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668753 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668995 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9p96\" (UniqueName: \"kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668995 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668995 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.668995 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.668929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.669209 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.669187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.669273 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.669244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.669324 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.669288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.671120 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.671099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.671246 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.671229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.679359 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.679338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9p96\" (UniqueName: \"kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96\") pod \"auth-test-model-4a426a-kserve-d66867b8c-jdjm5\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") " pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.780293 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.780266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:41:52.914289 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:52.914260    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:41:52.916099 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:41:52.916075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50d83717_ecba_4d0e_a81c_49b7bd3e107a.slice/crio-31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa WatchSource:0}: Error finding container 31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa: Status 404 returned error can't find the container with id 31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa
Jul 23 19:41:53.007385 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:53.007349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" event={"ID":"50d83717-ecba-4d0e-a81c-49b7bd3e107a","Type":"ContainerStarted","Data":"c78731e3a3848deebd243727e61dbe98159b689a2e866ff279277a87965de495"}
Jul 23 19:41:53.007521 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:53.007405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" event={"ID":"50d83717-ecba-4d0e-a81c-49b7bd3e107a","Type":"ContainerStarted","Data":"31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa"}
Jul 23 19:41:53.025296 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:53.025250    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" podStartSLOduration=1.025236375 podStartE2EDuration="1.025236375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:53.023846389 +0000 UTC m=+2455.926432461" watchObservedRunningTime="2026-07-23 19:41:53.025236375 +0000 UTC m=+2455.927822442"
Jul 23 19:41:56.020485 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:41:56.020453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:42:02.780536 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:02.780503    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:42:02.780968 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:02.780763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:42:02.792877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:02.792858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:42:03.070640 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:03.070557    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:42:21.726184 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:21.726144    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:42:37.266861 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:37.266802    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:42:54.823869 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:42:54.823772    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:43:13.317785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:43:13.317748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:43:30.814634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:43:30.814594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:43:49.222010 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:43:49.221971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:44:05.721863 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:05.721824    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:44:09.752770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:09.752742    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:44:10.547144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:10.547102    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:44:14.566189 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.566160    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:44:14.569618 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.569603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.572337 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.572308    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"sub-test-model-d5ee7b-kserve-self-signed-certs\""
Jul 23 19:44:14.579725 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.579702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:44:14.646294 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.646407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.646407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.646407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.646407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.646542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.646445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d75bm\" (UniqueName: \"kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747496 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d75bm\" (UniqueName: \"kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747835 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.747835 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.747702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.748024 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.748000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.748095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.748018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.748095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.748044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.749737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.749714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.750023 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.750005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.755314 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.755292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d75bm\" (UniqueName: \"kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm\") pod \"sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") " pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:14.879983 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:14.879908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:15.218323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:15.218296    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:44:15.219775 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:44:15.219748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda12fd142_f430_43d7_bbde_f6c6c02e0b7a.slice/crio-7439f932c88e5246b73230128c53d9b144c1c05a44c69dac584fed85d23fcb7e WatchSource:0}: Error finding container 7439f932c88e5246b73230128c53d9b144c1c05a44c69dac584fed85d23fcb7e: Status 404 returned error can't find the container with id 7439f932c88e5246b73230128c53d9b144c1c05a44c69dac584fed85d23fcb7e
Jul 23 19:44:15.770628 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:15.770591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" event={"ID":"a12fd142-f430-43d7-bbde-f6c6c02e0b7a","Type":"ContainerStarted","Data":"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"}
Jul 23 19:44:15.770628 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:15.770636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" event={"ID":"a12fd142-f430-43d7-bbde-f6c6c02e0b7a","Type":"ContainerStarted","Data":"7439f932c88e5246b73230128c53d9b144c1c05a44c69dac584fed85d23fcb7e"}
Jul 23 19:44:15.788552 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:15.788502    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" podStartSLOduration=1.788487646 podStartE2EDuration="1.788487646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:44:15.786025094 +0000 UTC m=+2598.688611193" watchObservedRunningTime="2026-07-23 19:44:15.788487646 +0000 UTC m=+2598.691073723"
Jul 23 19:44:21.843223 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:21.843185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:44:24.772389 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:24.772342    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" podUID="3ee7abd2-727f-430f-9490-7fe439b0597b" containerName="main" containerID="cri-o://cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c" gracePeriod=45
Jul 23 19:44:24.880303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:24.880268    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:24.880303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:24.880313    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:24.894039 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:24.894017    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:25.040891 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.040867    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:44:25.142563 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142525    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142602    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142629    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142738 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142984 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142742    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c94q9\" (UniqueName: \"kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9\") pod \"3ee7abd2-727f-430f-9490-7fe439b0597b\" (UID: \"3ee7abd2-727f-430f-9490-7fe439b0597b\") "
Jul 23 19:44:25.142984 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.142984 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142949    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.143144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.142996    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home" (OuterVolumeSpecName: "home") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.143144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.143038    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.143144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.143050    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.144949 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.144921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm" (OuterVolumeSpecName: "dshm") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.145063 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.145006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:44:25.145120 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.145079    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9" (OuterVolumeSpecName: "kube-api-access-c94q9") pod "3ee7abd2-727f-430f-9490-7fe439b0597b" (UID: "3ee7abd2-727f-430f-9490-7fe439b0597b"). InnerVolumeSpecName "kube-api-access-c94q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:44:25.243475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.243438    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c94q9\" (UniqueName: \"kubernetes.io/projected/3ee7abd2-727f-430f-9490-7fe439b0597b-kube-api-access-c94q9\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.243475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.243469    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee7abd2-727f-430f-9490-7fe439b0597b-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.243475 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.243482    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.243737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.243492    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee7abd2-727f-430f-9490-7fe439b0597b-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.567596 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.567551    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" podUID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" containerName="main" containerID="cri-o://c78731e3a3848deebd243727e61dbe98159b689a2e866ff279277a87965de495" gracePeriod=45
Jul 23 19:44:25.825239 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.825209    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ee7abd2-727f-430f-9490-7fe439b0597b" containerID="cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c" exitCode=0
Jul 23 19:44:25.825616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.825262    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"
Jul 23 19:44:25.825616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.825289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" event={"ID":"3ee7abd2-727f-430f-9490-7fe439b0597b","Type":"ContainerDied","Data":"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"}
Jul 23 19:44:25.825616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.825341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh" event={"ID":"3ee7abd2-727f-430f-9490-7fe439b0597b","Type":"ContainerDied","Data":"c664d76c6d9d6cb31e7bf750abe956b201f91332233f3b0c1a6e161eb774596d"}
Jul 23 19:44:25.825616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.825363    2563 scope.go:117] "RemoveContainer" containerID="cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"
Jul 23 19:44:25.826657 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.826637    2563 generic.go:356] "Generic (PLEG): container finished" podID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" containerID="c78731e3a3848deebd243727e61dbe98159b689a2e866ff279277a87965de495" exitCode=0
Jul 23 19:44:25.826762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.826713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" event={"ID":"50d83717-ecba-4d0e-a81c-49b7bd3e107a","Type":"ContainerDied","Data":"c78731e3a3848deebd243727e61dbe98159b689a2e866ff279277a87965de495"}
Jul 23 19:44:25.826762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.826749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5" event={"ID":"50d83717-ecba-4d0e-a81c-49b7bd3e107a","Type":"ContainerDied","Data":"31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa"}
Jul 23 19:44:25.826907 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.826764    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31cbbd1bf8078d9a8da213a83a4d23e4c62a8948ae9bf8c246bd886f5d9fdafa"
Jul 23 19:44:25.839347 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.839326    2563 scope.go:117] "RemoveContainer" containerID="cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"
Jul 23 19:44:25.839585 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.839554    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:44:25.839706 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:44:25.839613    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c\": container with ID starting with cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c not found: ID does not exist" containerID="cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"
Jul 23 19:44:25.839706 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.839666    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c"} err="failed to get container status \"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c\": rpc error: code = NotFound desc = could not find container \"cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c\": container with ID starting with cf7061848f7ef43178f1876b8fcf06972be83c46277a23e6d0f1510237fb052c not found: ID does not exist"
Jul 23 19:44:25.841222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.841204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:44:25.845535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.845515    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:44:25.851831 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.851797    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/auth-test-model-d5ee7b-kserve-c47c4d67-z8dqh"]
Jul 23 19:44:25.948617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.948617 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948615    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.948889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.948889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.948889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948722    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9p96\" (UniqueName: \"kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.948889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948747    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs\") pod \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\" (UID: \"50d83717-ecba-4d0e-a81c-49b7bd3e107a\") "
Jul 23 19:44:25.949098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home" (OuterVolumeSpecName: "home") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.949098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache" (OuterVolumeSpecName: "model-cache") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.949098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.948975    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:25.949207 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.949187    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.949257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.949204    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.949257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.949218    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:25.950978 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.950953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96" (OuterVolumeSpecName: "kube-api-access-q9p96") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "kube-api-access-q9p96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:44:25.951234 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.951205    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:44:25.951234 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:25.951214    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm" (OuterVolumeSpecName: "dshm") pod "50d83717-ecba-4d0e-a81c-49b7bd3e107a" (UID: "50d83717-ecba-4d0e-a81c-49b7bd3e107a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:44:26.050095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.049998    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d83717-ecba-4d0e-a81c-49b7bd3e107a-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:26.050095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.050040    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9p96\" (UniqueName: \"kubernetes.io/projected/50d83717-ecba-4d0e-a81c-49b7bd3e107a-kube-api-access-q9p96\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:26.050095 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.050052    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d83717-ecba-4d0e-a81c-49b7bd3e107a-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:44:26.833860 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.833836    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"
Jul 23 19:44:26.854318 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.854294    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:44:26.858052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:26.858031    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/auth-test-model-4a426a-kserve-d66867b8c-jdjm5"]
Jul 23 19:44:27.522629 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:27.522595    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ee7abd2-727f-430f-9490-7fe439b0597b" path="/var/lib/kubelet/pods/3ee7abd2-727f-430f-9490-7fe439b0597b/volumes"
Jul 23 19:44:27.523089 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:27.523070    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" path="/var/lib/kubelet/pods/50d83717-ecba-4d0e-a81c-49b7bd3e107a/volumes"
Jul 23 19:44:36.774533 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.774494    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:44:36.775078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775059    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" containerName="main"
Jul 23 19:44:36.775154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775080    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" containerName="main"
Jul 23 19:44:36.775154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775099    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ee7abd2-727f-430f-9490-7fe439b0597b" containerName="main"
Jul 23 19:44:36.775154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775109    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ee7abd2-727f-430f-9490-7fe439b0597b" containerName="main"
Jul 23 19:44:36.775301 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775214    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d83717-ecba-4d0e-a81c-49b7bd3e107a" containerName="main"
Jul 23 19:44:36.775301 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.775229    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ee7abd2-727f-430f-9490-7fe439b0597b" containerName="main"
Jul 23 19:44:36.778327 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.778306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.780455 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.780431    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"kube-root-ca.crt\""
Jul 23 19:44:36.781348 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.781320    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"sub-test-model-4a426a-kserve-self-signed-certs\""
Jul 23 19:44:36.781449 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.781351    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"openshift-service-ca.crt\""
Jul 23 19:44:36.781449 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.781414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"default-dockercfg-lqsjj\""
Jul 23 19:44:36.786433 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.786409    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:44:36.842170 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.842365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.842365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.842365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.842365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9hmk\" (UniqueName: \"kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.842500 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.842408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943171 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9hmk\" (UniqueName: \"kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943326 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943587 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943649 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943596    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.943712 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.943661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.945522 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.945502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.945842 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.945807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:36.950731 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:36.950710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9hmk\" (UniqueName: \"kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk\") pod \"sub-test-model-4a426a-kserve-6b5dddd655-5tpdt\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") " pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:37.089797 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:37.089710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:37.223977 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:37.223948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:44:37.225690 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:44:37.225654    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce2d34fd_29a9_467a_97ed_4b45fab79b3a.slice/crio-0a02f04f50152144cea284b381564c8920d3a5e53eaef0c11d46ca488a48236c WatchSource:0}: Error finding container 0a02f04f50152144cea284b381564c8920d3a5e53eaef0c11d46ca488a48236c: Status 404 returned error can't find the container with id 0a02f04f50152144cea284b381564c8920d3a5e53eaef0c11d46ca488a48236c
Jul 23 19:44:37.892531 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:37.892490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" event={"ID":"ce2d34fd-29a9-467a-97ed-4b45fab79b3a","Type":"ContainerStarted","Data":"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"}
Jul 23 19:44:37.892531 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:37.892533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" event={"ID":"ce2d34fd-29a9-467a-97ed-4b45fab79b3a","Type":"ContainerStarted","Data":"0a02f04f50152144cea284b381564c8920d3a5e53eaef0c11d46ca488a48236c"}
Jul 23 19:44:37.921990 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:37.921945    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" podStartSLOduration=1.9219275740000001 podStartE2EDuration="1.921927574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:44:37.91970919 +0000 UTC m=+2620.822295268" watchObservedRunningTime="2026-07-23 19:44:37.921927574 +0000 UTC m=+2620.824513650"
Jul 23 19:44:40.526277 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:40.526243    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:44:47.090447 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:47.090408    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:47.090878 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:47.090467    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:47.102503 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:47.102480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:47.967318 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:47.967284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:44:59.829060 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:44:59.829027    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:45:00.137448 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.137372    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"]
Jul 23 19:45:00.142098 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.142081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"
Jul 23 19:45:00.145194 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.145167    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:45:00.149999 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.149740    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"]
Jul 23 19:45:00.154094 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.154072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"
Jul 23 19:45:00.290784 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:00.290754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"]
Jul 23 19:45:00.292080 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:45:00.292055    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af0e9bc_5ab3_4a39_b08f_612f00aaaf59.slice/crio-2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9 WatchSource:0}: Error finding container 2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9: Status 404 returned error can't find the container with id 2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9
Jul 23 19:45:01.026944 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:01.026907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q" event={"ID":"2af0e9bc-5ab3-4a39-b08f-612f00aaaf59","Type":"ContainerStarted","Data":"7f5c033173aa5f90435596ac461b602bf8ae6acc04c1834282ca1fb5441926e9"}
Jul 23 19:45:01.027303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:01.026965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q" event={"ID":"2af0e9bc-5ab3-4a39-b08f-612f00aaaf59","Type":"ContainerStarted","Data":"2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9"}
Jul 23 19:45:01.042293 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:01.042240    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q" podStartSLOduration=1.04222614 podStartE2EDuration="1.04222614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:45:01.040864668 +0000 UTC m=+2643.943450770" watchObservedRunningTime="2026-07-23 19:45:01.04222614 +0000 UTC m=+2643.944812211"
Jul 23 19:45:02.033427 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:02.033396    2563 generic.go:356] "Generic (PLEG): container finished" podID="2af0e9bc-5ab3-4a39-b08f-612f00aaaf59" containerID="7f5c033173aa5f90435596ac461b602bf8ae6acc04c1834282ca1fb5441926e9" exitCode=0
Jul 23 19:45:02.033794 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:02.033438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q" event={"ID":"2af0e9bc-5ab3-4a39-b08f-612f00aaaf59","Type":"ContainerDied","Data":"7f5c033173aa5f90435596ac461b602bf8ae6acc04c1834282ca1fb5441926e9"}
Jul 23 19:45:03.172167 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:03.172144    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"
Jul 23 19:45:04.047367 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:04.047335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q"
Jul 23 19:45:04.047537 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:04.047381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-kwb2q" event={"ID":"2af0e9bc-5ab3-4a39-b08f-612f00aaaf59","Type":"ContainerDied","Data":"2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9"}
Jul 23 19:45:04.047537 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:04.047458    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ac22a7d03363c0d60970b93618ea3d4e89f1617e4b9235c0163792d64b487f9"
Jul 23 19:45:12.227860 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:12.227825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:45:17.587137 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:17.587104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:45:18.357573 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:18.357540    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:45:29.421630 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:29.421597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:45:32.606572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:32.606530    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" podUID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" containerName="main" containerID="cri-o://f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c" gracePeriod=45
Jul 23 19:45:32.868398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:32.868344    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:45:33.022368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022514 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022424    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022514 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022514 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d75bm\" (UniqueName: \"kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022514 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022728 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home\") pod \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\" (UID: \"a12fd142-f430-43d7-bbde-f6c6c02e0b7a\") "
Jul 23 19:45:33.022728 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022593    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.022728 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.022900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022835    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.022900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022854    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.022900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.022852    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home" (OuterVolumeSpecName: "home") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.024610 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.024579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm" (OuterVolumeSpecName: "dshm") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.024724 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.024607    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:45:33.024724 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.024650    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm" (OuterVolumeSpecName: "kube-api-access-d75bm") pod "a12fd142-f430-43d7-bbde-f6c6c02e0b7a" (UID: "a12fd142-f430-43d7-bbde-f6c6c02e0b7a"). InnerVolumeSpecName "kube-api-access-d75bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:45:33.123772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.123711    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.123772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.123734    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.123772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.123742    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.123772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.123750    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d75bm\" (UniqueName: \"kubernetes.io/projected/a12fd142-f430-43d7-bbde-f6c6c02e0b7a-kube-api-access-d75bm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.207922 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.207895    2563 generic.go:356] "Generic (PLEG): container finished" podID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" containerID="f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c" exitCode=0
Jul 23 19:45:33.208036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.207945    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"
Jul 23 19:45:33.208036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.207985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" event={"ID":"a12fd142-f430-43d7-bbde-f6c6c02e0b7a","Type":"ContainerDied","Data":"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"}
Jul 23 19:45:33.208036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.208031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs" event={"ID":"a12fd142-f430-43d7-bbde-f6c6c02e0b7a","Type":"ContainerDied","Data":"7439f932c88e5246b73230128c53d9b144c1c05a44c69dac584fed85d23fcb7e"}
Jul 23 19:45:33.208147 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.208049    2563 scope.go:117] "RemoveContainer" containerID="f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"
Jul 23 19:45:33.219504 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.219490    2563 scope.go:117] "RemoveContainer" containerID="f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"
Jul 23 19:45:33.219787 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:45:33.219771    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c\": container with ID starting with f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c not found: ID does not exist" containerID="f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"
Jul 23 19:45:33.219867 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.219794    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c"} err="failed to get container status \"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c\": rpc error: code = NotFound desc = could not find container \"f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c\": container with ID starting with f06906f953babd741fff0f1cd621291510e6cb8863ca4e06fbff8c07c8200f9c not found: ID does not exist"
Jul 23 19:45:33.232269 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.232247    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:45:33.235288 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.235256    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/sub-test-model-d5ee7b-kserve-d885fd4c9-ft2fs"]
Jul 23 19:45:33.377491 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.377396    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" podUID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" containerName="main" containerID="cri-o://84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e" gracePeriod=45
Jul 23 19:45:33.521633 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.521592    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" path="/var/lib/kubelet/pods/a12fd142-f430-43d7-bbde-f6c6c02e0b7a/volumes"
Jul 23 19:45:33.617211 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.617186    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:45:33.728532 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728456    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728540    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728575    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728609    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9hmk\" (UniqueName: \"kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728677 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache\") pod \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\" (UID: \"ce2d34fd-29a9-467a-97ed-4b45fab79b3a\") "
Jul 23 19:45:33.728963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728868    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home" (OuterVolumeSpecName: "home") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.728963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728883    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.729059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.728988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.730398 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.730378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:45:33.730537 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.730520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm" (OuterVolumeSpecName: "dshm") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:45:33.730625 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.730611    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk" (OuterVolumeSpecName: "kube-api-access-c9hmk") pod "ce2d34fd-29a9-467a-97ed-4b45fab79b3a" (UID: "ce2d34fd-29a9-467a-97ed-4b45fab79b3a"). InnerVolumeSpecName "kube-api-access-c9hmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:45:33.829632 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829609    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9hmk\" (UniqueName: \"kubernetes.io/projected/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-kube-api-access-c9hmk\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.829632 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829632    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.829774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829645    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.829774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829653    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.829774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829663    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:33.829774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:33.829670    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2d34fd-29a9-467a-97ed-4b45fab79b3a-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:45:34.219746 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.219714    2563 generic.go:356] "Generic (PLEG): container finished" podID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" containerID="84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e" exitCode=0
Jul 23 19:45:34.219954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.219766    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"
Jul 23 19:45:34.219954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.219795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" event={"ID":"ce2d34fd-29a9-467a-97ed-4b45fab79b3a","Type":"ContainerDied","Data":"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"}
Jul 23 19:45:34.219954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.219860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt" event={"ID":"ce2d34fd-29a9-467a-97ed-4b45fab79b3a","Type":"ContainerDied","Data":"0a02f04f50152144cea284b381564c8920d3a5e53eaef0c11d46ca488a48236c"}
Jul 23 19:45:34.219954 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.219882    2563 scope.go:117] "RemoveContainer" containerID="84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"
Jul 23 19:45:34.229469 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.229452    2563 scope.go:117] "RemoveContainer" containerID="84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"
Jul 23 19:45:34.229712 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:45:34.229694    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e\": container with ID starting with 84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e not found: ID does not exist" containerID="84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"
Jul 23 19:45:34.229775 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.229719    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e"} err="failed to get container status \"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e\": rpc error: code = NotFound desc = could not find container \"84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e\": container with ID starting with 84fcf88caf6094e5369bb497ca440dc8cd049b2540d4fd237565048bae2a880e not found: ID does not exist"
Jul 23 19:45:34.239768 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.239745    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:45:34.243165 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:34.243144    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/sub-test-model-4a426a-kserve-6b5dddd655-5tpdt"]
Jul 23 19:45:35.076067 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076039    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076425    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" containerName="main"
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076437    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" containerName="main"
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076453    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af0e9bc-5ab3-4a39-b08f-612f00aaaf59" containerName="cleanup"
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076459    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af0e9bc-5ab3-4a39-b08f-612f00aaaf59" containerName="cleanup"
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076471    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" containerName="main"
Jul 23 19:45:35.076479 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076476    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" containerName="main"
Jul 23 19:45:35.076702 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076546    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af0e9bc-5ab3-4a39-b08f-612f00aaaf59" containerName="cleanup"
Jul 23 19:45:35.076702 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076554    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a12fd142-f430-43d7-bbde-f6c6c02e0b7a" containerName="main"
Jul 23 19:45:35.076702 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.076562    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" containerName="main"
Jul 23 19:45:35.079759 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.079740    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.082127 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.082104    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"openshift-service-ca.crt\""
Jul 23 19:45:35.082896 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.082875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"kube-root-ca.crt\""
Jul 23 19:45:35.083017 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.082886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"rate-test-model-d5ee7b-kserve-self-signed-certs\""
Jul 23 19:45:35.083162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.083147    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d5ee7b\"/\"default-dockercfg-bgfz6\""
Jul 23 19:45:35.089372 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.089353    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:45:35.241096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.241275 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58xtl\" (UniqueName: \"kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.241275 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.241275 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.241275 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.241439 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.241318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342341 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xtl\" (UniqueName: \"kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342536 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342536 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342536 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342536 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342733 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342792 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.342876 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.342850    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.344612 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.344583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.344863 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.344847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.350157 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.350135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xtl\" (UniqueName: \"kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl\") pod \"rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") " pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.389292 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.389269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:35.524676 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.524639    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce2d34fd-29a9-467a-97ed-4b45fab79b3a" path="/var/lib/kubelet/pods/ce2d34fd-29a9-467a-97ed-4b45fab79b3a/volumes"
Jul 23 19:45:35.545084 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:35.545055    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:45:35.546197 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:45:35.546173    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74859b76_ebed_4481_81c1_0bc59faa6da9.slice/crio-8f52aba59c0ad7c99f551eff94ac303db46e7ed61fdab525ff20a53434a10f8a WatchSource:0}: Error finding container 8f52aba59c0ad7c99f551eff94ac303db46e7ed61fdab525ff20a53434a10f8a: Status 404 returned error can't find the container with id 8f52aba59c0ad7c99f551eff94ac303db46e7ed61fdab525ff20a53434a10f8a
Jul 23 19:45:36.234805 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:36.234771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" event={"ID":"74859b76-ebed-4481-81c1-0bc59faa6da9","Type":"ContainerStarted","Data":"14f44ea02e57fe4217b31ca1623a428a064ad84293d23e478eb682c1805eb339"}
Jul 23 19:45:36.234805 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:36.234825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" event={"ID":"74859b76-ebed-4481-81c1-0bc59faa6da9","Type":"ContainerStarted","Data":"8f52aba59c0ad7c99f551eff94ac303db46e7ed61fdab525ff20a53434a10f8a"}
Jul 23 19:45:36.259108 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:36.259062    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" podStartSLOduration=1.259044639 podStartE2EDuration="1.259044639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:45:36.257875465 +0000 UTC m=+2679.160461543" watchObservedRunningTime="2026-07-23 19:45:36.259044639 +0000 UTC m=+2679.161630738"
Jul 23 19:45:37.527488 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:37.527455    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:45:45.390162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:45.390131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:45.390634 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:45.390177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:45.402290 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:45.402271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:46.300523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:46.300494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:45:57.885746 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:45:57.885677    2563 scope.go:117] "RemoveContainer" containerID="378e768a74d4880296f105c92edfc043da73803388800598b163e12dd2c46a44"
Jul 23 19:46:03.206680 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.206651    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:46:03.211486 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.211455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.214506 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.214475    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"kube-root-ca.crt\""
Jul 23 19:46:03.214623 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.214485    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"default-dockercfg-lqsjj\""
Jul 23 19:46:03.215200 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.215172    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"openshift-service-ca.crt\""
Jul 23 19:46:03.215323 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.215210    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a426a\"/\"rate-test-model-4a426a-kserve-self-signed-certs\""
Jul 23 19:46:03.226762 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.226740    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:46:03.248094 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.248206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhb64\" (UniqueName: \"kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.248206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.248206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.248325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.248325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.248285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349389 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhb64\" (UniqueName: \"kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349540 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349877 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.349982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.349884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.351788 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.351767    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.351935 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.351916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.359034 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.359010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhb64\" (UniqueName: \"kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64\") pod \"rate-test-model-4a426a-kserve-667df94c4c-zz4hv\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") " pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.520772 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.520743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:03.658059 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:03.658020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:46:03.662629 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:46:03.662605    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89ebf26a_5720_4e9b_b134_88d59998d27b.slice/crio-d89bb5d11603001305fa7bca8c9e26efb41a3d5fa9be481c4ecdc242fdad4034 WatchSource:0}: Error finding container d89bb5d11603001305fa7bca8c9e26efb41a3d5fa9be481c4ecdc242fdad4034: Status 404 returned error can't find the container with id d89bb5d11603001305fa7bca8c9e26efb41a3d5fa9be481c4ecdc242fdad4034
Jul 23 19:46:04.394344 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:04.394311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" event={"ID":"89ebf26a-5720-4e9b-b134-88d59998d27b","Type":"ContainerStarted","Data":"4534fcf76d41a56f8f050adfd7fd6793db61a4a8d37afdaf4a0265098ca7e40d"}
Jul 23 19:46:04.394344 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:04.394349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" event={"ID":"89ebf26a-5720-4e9b-b134-88d59998d27b","Type":"ContainerStarted","Data":"d89bb5d11603001305fa7bca8c9e26efb41a3d5fa9be481c4ecdc242fdad4034"}
Jul 23 19:46:04.412531 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:04.412484    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" podStartSLOduration=1.412470458 podStartE2EDuration="1.412470458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:46:04.411616366 +0000 UTC m=+2707.314202453" watchObservedRunningTime="2026-07-23 19:46:04.412470458 +0000 UTC m=+2707.315056533"
Jul 23 19:46:06.777084 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:06.777052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:46:13.521332 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:13.521301    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:13.521699 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:13.521350    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:13.533755 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:13.533731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:14.460832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:14.460792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:46:28.328850 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:28.328805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:46:43.464652 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:43.464616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:46:49.553735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:49.553702    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:46:50.495534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:50.495502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:46:51.430560 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.430525    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910760368"]
Jul 23 19:46:51.434109 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.434093    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:51.436986 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.436972    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kmbt9\""
Jul 23 19:46:51.444856 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.444801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910760368"]
Jul 23 19:46:51.533063 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.533035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9629\" (UniqueName: \"kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629\") pod \"test-curl-7751-140336910760368\" (UID: \"c772abbf-32ba-440a-89b6-17176a89bdb7\") " pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:51.633853 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.633823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9629\" (UniqueName: \"kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629\") pod \"test-curl-7751-140336910760368\" (UID: \"c772abbf-32ba-440a-89b6-17176a89bdb7\") " pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:51.643888 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.643864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9629\" (UniqueName: \"kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629\") pod \"test-curl-7751-140336910760368\" (UID: \"c772abbf-32ba-440a-89b6-17176a89bdb7\") " pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:51.743553 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.743489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:51.876076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.876051    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910760368"]
Jul 23 19:46:51.877888 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:46:51.877860    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc772abbf_32ba_440a_89b6_17176a89bdb7.slice/crio-608f641dd97bfc675addfa91817f1c3effcb30151a692254bc817ebe3342fb63 WatchSource:0}: Error finding container 608f641dd97bfc675addfa91817f1c3effcb30151a692254bc817ebe3342fb63: Status 404 returned error can't find the container with id 608f641dd97bfc675addfa91817f1c3effcb30151a692254bc817ebe3342fb63
Jul 23 19:46:51.879191 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:51.879171    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:46:52.646219 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:52.646192    2563 generic.go:356] "Generic (PLEG): container finished" podID="c772abbf-32ba-440a-89b6-17176a89bdb7" containerID="2b3b67260c1c2ed582cc075878467da11fc88203f7eb7ab1aff2eec326e0a99c" exitCode=0
Jul 23 19:46:52.646516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:52.646235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910760368" event={"ID":"c772abbf-32ba-440a-89b6-17176a89bdb7","Type":"ContainerDied","Data":"2b3b67260c1c2ed582cc075878467da11fc88203f7eb7ab1aff2eec326e0a99c"}
Jul 23 19:46:52.646516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:52.646266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910760368" event={"ID":"c772abbf-32ba-440a-89b6-17176a89bdb7","Type":"ContainerStarted","Data":"608f641dd97bfc675addfa91817f1c3effcb30151a692254bc817ebe3342fb63"}
Jul 23 19:46:53.783696 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.783673    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:53.820516 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.820492    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910760368_c772abbf-32ba-440a-89b6-17176a89bdb7/test-curl-7751-140336910760368/0.log"
Jul 23 19:46:53.843202 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.843181    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910760368"]
Jul 23 19:46:53.848350 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.848325    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910760368"]
Jul 23 19:46:53.853872 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.853852    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9629\" (UniqueName: \"kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629\") pod \"c772abbf-32ba-440a-89b6-17176a89bdb7\" (UID: \"c772abbf-32ba-440a-89b6-17176a89bdb7\") "
Jul 23 19:46:53.855993 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.855973    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629" (OuterVolumeSpecName: "kube-api-access-p9629") pod "c772abbf-32ba-440a-89b6-17176a89bdb7" (UID: "c772abbf-32ba-440a-89b6-17176a89bdb7"). InnerVolumeSpecName "kube-api-access-p9629". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:46:53.954482 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.954417    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9629\" (UniqueName: \"kubernetes.io/projected/c772abbf-32ba-440a-89b6-17176a89bdb7-kube-api-access-p9629\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:46:53.969365 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.969339    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910761520"]
Jul 23 19:46:53.969728 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.969717    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c772abbf-32ba-440a-89b6-17176a89bdb7" containerName="test-curl-7751-140336910760368"
Jul 23 19:46:53.969771 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.969730    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c772abbf-32ba-440a-89b6-17176a89bdb7" containerName="test-curl-7751-140336910760368"
Jul 23 19:46:53.969889 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.969793    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c772abbf-32ba-440a-89b6-17176a89bdb7" containerName="test-curl-7751-140336910760368"
Jul 23 19:46:53.973002 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.972984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:53.978338 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:53.978293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910761520"]
Jul 23 19:46:54.055680 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.055657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl7tp\" (UniqueName: \"kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp\") pod \"test-curl-7751-140336910761520\" (UID: \"b467283e-b792-438f-8d5c-be5a5682546f\") " pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:54.157045 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.157019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl7tp\" (UniqueName: \"kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp\") pod \"test-curl-7751-140336910761520\" (UID: \"b467283e-b792-438f-8d5c-be5a5682546f\") " pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:54.166721 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.166696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl7tp\" (UniqueName: \"kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp\") pod \"test-curl-7751-140336910761520\" (UID: \"b467283e-b792-438f-8d5c-be5a5682546f\") " pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:54.284048 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.284023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:54.416680 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.416656    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910761520"]
Jul 23 19:46:54.418346 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:46:54.418318    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb467283e_b792_438f_8d5c_be5a5682546f.slice/crio-47541afba3f563451b568c489398111773b0890614415815d8b15721d16bdb9d WatchSource:0}: Error finding container 47541afba3f563451b568c489398111773b0890614415815d8b15721d16bdb9d: Status 404 returned error can't find the container with id 47541afba3f563451b568c489398111773b0890614415815d8b15721d16bdb9d
Jul 23 19:46:54.658939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.658849    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:54.658939 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.658866    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="608f641dd97bfc675addfa91817f1c3effcb30151a692254bc817ebe3342fb63"
Jul 23 19:46:54.660064 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:54.660040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910761520" event={"ID":"b467283e-b792-438f-8d5c-be5a5682546f","Type":"ContainerStarted","Data":"47541afba3f563451b568c489398111773b0890614415815d8b15721d16bdb9d"}
Jul 23 19:46:54.669086 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:46:54.669064    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7751-140336910760368\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="c772abbf-32ba-440a-89b6-17176a89bdb7" pod="opendatahub/test-curl-7751-140336910760368"
Jul 23 19:46:55.526640 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:55.526609    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c772abbf-32ba-440a-89b6-17176a89bdb7" path="/var/lib/kubelet/pods/c772abbf-32ba-440a-89b6-17176a89bdb7/volumes"
Jul 23 19:46:55.666013 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:55.665977    2563 generic.go:356] "Generic (PLEG): container finished" podID="b467283e-b792-438f-8d5c-be5a5682546f" containerID="ac16d330f6c27b4539dcbbb1d2150656bd1547cb560136cc67a513903ddea3ff" exitCode=0
Jul 23 19:46:55.666174 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:55.666016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910761520" event={"ID":"b467283e-b792-438f-8d5c-be5a5682546f","Type":"ContainerDied","Data":"ac16d330f6c27b4539dcbbb1d2150656bd1547cb560136cc67a513903ddea3ff"}
Jul 23 19:46:56.805752 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.805728    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:56.829258 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.829233    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910761520_b467283e-b792-438f-8d5c-be5a5682546f/test-curl-7751-140336910761520/0.log"
Jul 23 19:46:56.853982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.853956    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910761520"]
Jul 23 19:46:56.860632 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.860603    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910761520"]
Jul 23 19:46:56.879900 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.879877    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl7tp\" (UniqueName: \"kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp\") pod \"b467283e-b792-438f-8d5c-be5a5682546f\" (UID: \"b467283e-b792-438f-8d5c-be5a5682546f\") "
Jul 23 19:46:56.881857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.881833    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp" (OuterVolumeSpecName: "kube-api-access-wl7tp") pod "b467283e-b792-438f-8d5c-be5a5682546f" (UID: "b467283e-b792-438f-8d5c-be5a5682546f"). InnerVolumeSpecName "kube-api-access-wl7tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:46:56.981319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:56.981263    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl7tp\" (UniqueName: \"kubernetes.io/projected/b467283e-b792-438f-8d5c-be5a5682546f-kube-api-access-wl7tp\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:46:57.015076 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.015049    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:57.015435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.015422    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b467283e-b792-438f-8d5c-be5a5682546f" containerName="test-curl-7751-140336910761520"
Jul 23 19:46:57.015476 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.015439    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b467283e-b792-438f-8d5c-be5a5682546f" containerName="test-curl-7751-140336910761520"
Jul 23 19:46:57.015524 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.015514    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b467283e-b792-438f-8d5c-be5a5682546f" containerName="test-curl-7751-140336910761520"
Jul 23 19:46:57.018591 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.018576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:57.024036 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.024012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:57.182884 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.182854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xkd8\" (UniqueName: \"kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8\") pod \"test-curl-7751-140336910918544\" (UID: \"e4b1d750-88be-4820-8e60-feddbefc7679\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:57.284320 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.284294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xkd8\" (UniqueName: \"kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8\") pod \"test-curl-7751-140336910918544\" (UID: \"e4b1d750-88be-4820-8e60-feddbefc7679\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:57.292596 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.292572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xkd8\" (UniqueName: \"kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8\") pod \"test-curl-7751-140336910918544\" (UID: \"e4b1d750-88be-4820-8e60-feddbefc7679\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:57.328324 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.328299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:57.456268 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.456241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:57.457699 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:46:57.457673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4b1d750_88be_4820_8e60_feddbefc7679.slice/crio-8abd8fbfe1f8678af1d65e5f2845e421c81e1373ac8c84330f72f0f28a6d8d31 WatchSource:0}: Error finding container 8abd8fbfe1f8678af1d65e5f2845e421c81e1373ac8c84330f72f0f28a6d8d31: Status 404 returned error can't find the container with id 8abd8fbfe1f8678af1d65e5f2845e421c81e1373ac8c84330f72f0f28a6d8d31
Jul 23 19:46:57.522303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.522281    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b467283e-b792-438f-8d5c-be5a5682546f" path="/var/lib/kubelet/pods/b467283e-b792-438f-8d5c-be5a5682546f/volumes"
Jul 23 19:46:57.677910 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.677802    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910761520"
Jul 23 19:46:57.677910 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.677838    2563 scope.go:117] "RemoveContainer" containerID="ac16d330f6c27b4539dcbbb1d2150656bd1547cb560136cc67a513903ddea3ff"
Jul 23 19:46:57.679087 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.679064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910918544" event={"ID":"e4b1d750-88be-4820-8e60-feddbefc7679","Type":"ContainerStarted","Data":"8abd8fbfe1f8678af1d65e5f2845e421c81e1373ac8c84330f72f0f28a6d8d31"}
Jul 23 19:46:57.914270 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.914250    2563 scope.go:117] "RemoveContainer" containerID="61d560c5b7bf504acee5ec476437f968a81de3ac5af32d6a34a30cbc2fe20c37"
Jul 23 19:46:57.922571 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:57.922552    2563 scope.go:117] "RemoveContainer" containerID="4f67a3663e7a9233a561c4bc560c2b843947ca8f418ae40a3db50c58a6d66d8a"
Jul 23 19:46:58.685535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:58.685498    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4b1d750-88be-4820-8e60-feddbefc7679" containerID="7ef6cc40149ec7b602243d50cec57488571422a13de42887de85ab79a8693932" exitCode=0
Jul 23 19:46:58.685707 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:58.685583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910918544" event={"ID":"e4b1d750-88be-4820-8e60-feddbefc7679","Type":"ContainerDied","Data":"7ef6cc40149ec7b602243d50cec57488571422a13de42887de85ab79a8693932"}
Jul 23 19:46:59.419401 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.419367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:46:59.821582 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.821561    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:46:59.847038 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.847014    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910918544_e4b1d750-88be-4820-8e60-feddbefc7679/test-curl-7751-140336910918544/0.log"
Jul 23 19:46:59.873260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.873237    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:59.880325 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.880299    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:59.907216 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.907197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xkd8\" (UniqueName: \"kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8\") pod \"e4b1d750-88be-4820-8e60-feddbefc7679\" (UID: \"e4b1d750-88be-4820-8e60-feddbefc7679\") "
Jul 23 19:46:59.909220 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.909202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8" (OuterVolumeSpecName: "kube-api-access-9xkd8") pod "e4b1d750-88be-4820-8e60-feddbefc7679" (UID: "e4b1d750-88be-4820-8e60-feddbefc7679"). InnerVolumeSpecName "kube-api-access-9xkd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:46:59.995056 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.994993    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:46:59.995364 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.995353    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4b1d750-88be-4820-8e60-feddbefc7679" containerName="test-curl-7751-140336910918544"
Jul 23 19:46:59.995405 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.995366    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4b1d750-88be-4820-8e60-feddbefc7679" containerName="test-curl-7751-140336910918544"
Jul 23 19:46:59.995436 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.995428    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4b1d750-88be-4820-8e60-feddbefc7679" containerName="test-curl-7751-140336910918544"
Jul 23 19:46:59.998429 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:46:59.998415    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.006007 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.005987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:47:00.008027 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.008006    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xkd8\" (UniqueName: \"kubernetes.io/projected/e4b1d750-88be-4820-8e60-feddbefc7679-kube-api-access-9xkd8\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:00.109178 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.109151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj75j\" (UniqueName: \"kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j\") pod \"test-curl-7751-140336910918544\" (UID: \"72259e35-5857-4b4a-94c1-b980c4dbc2b5\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.209921 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.209884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj75j\" (UniqueName: \"kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j\") pod \"test-curl-7751-140336910918544\" (UID: \"72259e35-5857-4b4a-94c1-b980c4dbc2b5\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.218992 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.218975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj75j\" (UniqueName: \"kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j\") pod \"test-curl-7751-140336910918544\" (UID: \"72259e35-5857-4b4a-94c1-b980c4dbc2b5\") " pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.307982 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.307960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.440505 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.440481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:47:00.441682 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:47:00.441655    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72259e35_5857_4b4a_94c1_b980c4dbc2b5.slice/crio-455263ce6d855375786d936e49991c47ad3dd2769dd5c7e6805eb2952b58b816 WatchSource:0}: Error finding container 455263ce6d855375786d936e49991c47ad3dd2769dd5c7e6805eb2952b58b816: Status 404 returned error can't find the container with id 455263ce6d855375786d936e49991c47ad3dd2769dd5c7e6805eb2952b58b816
Jul 23 19:47:00.697222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.697150    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:00.697222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.697188    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8abd8fbfe1f8678af1d65e5f2845e421c81e1373ac8c84330f72f0f28a6d8d31"
Jul 23 19:47:00.698413 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.698387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910918544" event={"ID":"72259e35-5857-4b4a-94c1-b980c4dbc2b5","Type":"ContainerStarted","Data":"455263ce6d855375786d936e49991c47ad3dd2769dd5c7e6805eb2952b58b816"}
Jul 23 19:47:00.708770 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:00.708744    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7751-140336910918544" oldPodUID="e4b1d750-88be-4820-8e60-feddbefc7679" podUID="72259e35-5857-4b4a-94c1-b980c4dbc2b5"
Jul 23 19:47:01.521918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:01.521887    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4b1d750-88be-4820-8e60-feddbefc7679" path="/var/lib/kubelet/pods/e4b1d750-88be-4820-8e60-feddbefc7679/volumes"
Jul 23 19:47:01.705331 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:01.705301    2563 generic.go:356] "Generic (PLEG): container finished" podID="72259e35-5857-4b4a-94c1-b980c4dbc2b5" containerID="19ef40fa8649a1eb74660e247c8feca4376fff6c0ecdb67bacd037f1304c1b66" exitCode=0
Jul 23 19:47:01.705464 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:01.705387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910918544" event={"ID":"72259e35-5857-4b4a-94c1-b980c4dbc2b5","Type":"ContainerDied","Data":"19ef40fa8649a1eb74660e247c8feca4376fff6c0ecdb67bacd037f1304c1b66"}
Jul 23 19:47:02.847393 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.847369    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:02.868652 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.868625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910918544_72259e35-5857-4b4a-94c1-b980c4dbc2b5/test-curl-7751-140336910918544/0.log"
Jul 23 19:47:02.892416 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.892387    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:47:02.895470 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.895448    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910918544"]
Jul 23 19:47:02.931127 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.931101    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj75j\" (UniqueName: \"kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j\") pod \"72259e35-5857-4b4a-94c1-b980c4dbc2b5\" (UID: \"72259e35-5857-4b4a-94c1-b980c4dbc2b5\") "
Jul 23 19:47:02.933080 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:02.933063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j" (OuterVolumeSpecName: "kube-api-access-bj75j") pod "72259e35-5857-4b4a-94c1-b980c4dbc2b5" (UID: "72259e35-5857-4b4a-94c1-b980c4dbc2b5"). InnerVolumeSpecName "kube-api-access-bj75j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:03.020080 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.020057    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:03.020453 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.020441    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72259e35-5857-4b4a-94c1-b980c4dbc2b5" containerName="test-curl-7751-140336910918544"
Jul 23 19:47:03.020498 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.020454    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72259e35-5857-4b4a-94c1-b980c4dbc2b5" containerName="test-curl-7751-140336910918544"
Jul 23 19:47:03.020534 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.020517    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72259e35-5857-4b4a-94c1-b980c4dbc2b5" containerName="test-curl-7751-140336910918544"
Jul 23 19:47:03.023487 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.023472    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:03.032061 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.032039    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:03.032162 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.032058    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj75j\" (UniqueName: \"kubernetes.io/projected/72259e35-5857-4b4a-94c1-b980c4dbc2b5-kube-api-access-bj75j\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:03.133018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.132994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhwl\" (UniqueName: \"kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl\") pod \"test-curl-7751-140336910927312\" (UID: \"61ccaf81-2afb-4f4d-a184-a8433c4493ce\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:03.233780 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.233751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhwl\" (UniqueName: \"kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl\") pod \"test-curl-7751-140336910927312\" (UID: \"61ccaf81-2afb-4f4d-a184-a8433c4493ce\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:03.243303 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.243281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhwl\" (UniqueName: \"kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl\") pod \"test-curl-7751-140336910927312\" (UID: \"61ccaf81-2afb-4f4d-a184-a8433c4493ce\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:03.332463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.332371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:03.466056 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.466012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:03.466794 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:47:03.466770    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61ccaf81_2afb_4f4d_a184_a8433c4493ce.slice/crio-7314a02f40fadccc22f4cb9cd75d5b86f1e5f7df6ace8e818bf81fa21f028894 WatchSource:0}: Error finding container 7314a02f40fadccc22f4cb9cd75d5b86f1e5f7df6ace8e818bf81fa21f028894: Status 404 returned error can't find the container with id 7314a02f40fadccc22f4cb9cd75d5b86f1e5f7df6ace8e818bf81fa21f028894
Jul 23 19:47:03.522614 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.522588    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72259e35-5857-4b4a-94c1-b980c4dbc2b5" path="/var/lib/kubelet/pods/72259e35-5857-4b4a-94c1-b980c4dbc2b5/volumes"
Jul 23 19:47:03.717904 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.717829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910927312" event={"ID":"61ccaf81-2afb-4f4d-a184-a8433c4493ce","Type":"ContainerStarted","Data":"7314a02f40fadccc22f4cb9cd75d5b86f1e5f7df6ace8e818bf81fa21f028894"}
Jul 23 19:47:03.719079 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.719059    2563 scope.go:117] "RemoveContainer" containerID="19ef40fa8649a1eb74660e247c8feca4376fff6c0ecdb67bacd037f1304c1b66"
Jul 23 19:47:03.719079 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:03.719068    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910918544"
Jul 23 19:47:04.573212 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.573180    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" podUID="74859b76-ebed-4481-81c1-0bc59faa6da9" containerName="main" containerID="cri-o://14f44ea02e57fe4217b31ca1623a428a064ad84293d23e478eb682c1805eb339" gracePeriod=45
Jul 23 19:47:04.726705 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.726673    2563 generic.go:356] "Generic (PLEG): container finished" podID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" containerID="c0e193ff13648471ec48c99b78a17b533cc7aa98d2d80dcd28400a15afb60b70" exitCode=0
Jul 23 19:47:04.726924 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.726761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910927312" event={"ID":"61ccaf81-2afb-4f4d-a184-a8433c4493ce","Type":"ContainerDied","Data":"c0e193ff13648471ec48c99b78a17b533cc7aa98d2d80dcd28400a15afb60b70"}
Jul 23 19:47:04.728511 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.728487    2563 generic.go:356] "Generic (PLEG): container finished" podID="74859b76-ebed-4481-81c1-0bc59faa6da9" containerID="14f44ea02e57fe4217b31ca1623a428a064ad84293d23e478eb682c1805eb339" exitCode=0
Jul 23 19:47:04.728628 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.728516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" event={"ID":"74859b76-ebed-4481-81c1-0bc59faa6da9","Type":"ContainerDied","Data":"14f44ea02e57fe4217b31ca1623a428a064ad84293d23e478eb682c1805eb339"}
Jul 23 19:47:04.833407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.833353    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:47:04.944976 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.944948    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.944993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945073    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945124 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945321 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58xtl\" (UniqueName: \"kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl\") pod \"74859b76-ebed-4481-81c1-0bc59faa6da9\" (UID: \"74859b76-ebed-4481-81c1-0bc59faa6da9\") "
Jul 23 19:47:04.945321 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945212    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache" (OuterVolumeSpecName: "model-cache") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:04.945321 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:04.945321 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home" (OuterVolumeSpecName: "home") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:04.945452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945421    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:04.945452 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945440    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:04.945554 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.945453    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:04.947073 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.947047    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm" (OuterVolumeSpecName: "dshm") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:04.947181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.947135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:47:04.947222 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:04.947169    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl" (OuterVolumeSpecName: "kube-api-access-58xtl") pod "74859b76-ebed-4481-81c1-0bc59faa6da9" (UID: "74859b76-ebed-4481-81c1-0bc59faa6da9"). InnerVolumeSpecName "kube-api-access-58xtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:05.045835 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.045793    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74859b76-ebed-4481-81c1-0bc59faa6da9-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:05.045958 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.045840    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74859b76-ebed-4481-81c1-0bc59faa6da9-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:05.045958 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.045856    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58xtl\" (UniqueName: \"kubernetes.io/projected/74859b76-ebed-4481-81c1-0bc59faa6da9-kube-api-access-58xtl\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:05.514650 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.514614    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" podUID="89ebf26a-5720-4e9b-b134-88d59998d27b" containerName="main" containerID="cri-o://4534fcf76d41a56f8f050adfd7fd6793db61a4a8d37afdaf4a0265098ca7e40d" gracePeriod=45
Jul 23 19:47:05.738185 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.738160    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"
Jul 23 19:47:05.738614 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.738196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv" event={"ID":"74859b76-ebed-4481-81c1-0bc59faa6da9","Type":"ContainerDied","Data":"8f52aba59c0ad7c99f551eff94ac303db46e7ed61fdab525ff20a53434a10f8a"}
Jul 23 19:47:05.738614 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.738256    2563 scope.go:117] "RemoveContainer" containerID="14f44ea02e57fe4217b31ca1623a428a064ad84293d23e478eb682c1805eb339"
Jul 23 19:47:05.740145 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.740121    2563 generic.go:356] "Generic (PLEG): container finished" podID="89ebf26a-5720-4e9b-b134-88d59998d27b" containerID="4534fcf76d41a56f8f050adfd7fd6793db61a4a8d37afdaf4a0265098ca7e40d" exitCode=0
Jul 23 19:47:05.740265 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.740212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" event={"ID":"89ebf26a-5720-4e9b-b134-88d59998d27b","Type":"ContainerDied","Data":"4534fcf76d41a56f8f050adfd7fd6793db61a4a8d37afdaf4a0265098ca7e40d"}
Jul 23 19:47:05.774052 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.774029    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:47:05.780913 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.780891    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d5ee7b/rate-test-model-d5ee7b-kserve-5dbdfff58c-mgjlv"]
Jul 23 19:47:05.795891 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.795804    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:47:05.897260 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.897240    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:05.918675 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.918648    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910927312_61ccaf81-2afb-4f4d-a184-a8433c4493ce/test-curl-7751-140336910927312/0.log"
Jul 23 19:47:05.944160 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.944132    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhb64\" (UniqueName: \"kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955149    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955182    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.955357    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir\") pod \"89ebf26a-5720-4e9b-b134-88d59998d27b\" (UID: \"89ebf26a-5720-4e9b-b134-88d59998d27b\") "
Jul 23 19:47:05.958263 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.956053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:05.959114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.958842    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:05.959114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.958933    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache" (OuterVolumeSpecName: "model-cache") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:05.959256 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.959128    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home" (OuterVolumeSpecName: "home") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:05.959256 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.959134    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64" (OuterVolumeSpecName: "kube-api-access-mhb64") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "kube-api-access-mhb64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:05.964134 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.964105    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm" (OuterVolumeSpecName: "dshm") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:47:05.964223 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:05.964203    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89ebf26a-5720-4e9b-b134-88d59998d27b" (UID: "89ebf26a-5720-4e9b-b134-88d59998d27b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:47:06.056764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056701    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qhwl\" (UniqueName: \"kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl\") pod \"61ccaf81-2afb-4f4d-a184-a8433c4493ce\" (UID: \"61ccaf81-2afb-4f4d-a184-a8433c4493ce\") "
Jul 23 19:47:06.056895 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056882    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhb64\" (UniqueName: \"kubernetes.io/projected/89ebf26a-5720-4e9b-b134-88d59998d27b-kube-api-access-mhb64\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.056936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056894    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.056936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056905    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.056936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056912    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ebf26a-5720-4e9b-b134-88d59998d27b-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.056936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056918    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.056936 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.056926    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89ebf26a-5720-4e9b-b134-88d59998d27b-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.058549 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.058529    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl" (OuterVolumeSpecName: "kube-api-access-5qhwl") pod "61ccaf81-2afb-4f4d-a184-a8433c4493ce" (UID: "61ccaf81-2afb-4f4d-a184-a8433c4493ce"). InnerVolumeSpecName "kube-api-access-5qhwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:06.075747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.075723    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:06.076115 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076102    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74859b76-ebed-4481-81c1-0bc59faa6da9" containerName="main"
Jul 23 19:47:06.076115 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076116    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="74859b76-ebed-4481-81c1-0bc59faa6da9" containerName="main"
Jul 23 19:47:06.076192 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076122    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ebf26a-5720-4e9b-b134-88d59998d27b" containerName="main"
Jul 23 19:47:06.076192 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076128    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ebf26a-5720-4e9b-b134-88d59998d27b" containerName="main"
Jul 23 19:47:06.076192 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076142    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:06.076192 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076148    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:06.076319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076211    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:06.076319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076221    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="74859b76-ebed-4481-81c1-0bc59faa6da9" containerName="main"
Jul 23 19:47:06.076319 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.076228    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ebf26a-5720-4e9b-b134-88d59998d27b" containerName="main"
Jul 23 19:47:06.079259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.079245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.089128 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.089105    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:06.157493 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.157467    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qhwl\" (UniqueName: \"kubernetes.io/projected/61ccaf81-2afb-4f4d-a184-a8433c4493ce-kube-api-access-5qhwl\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:06.257778 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.257750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87vpr\" (UniqueName: \"kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr\") pod \"test-curl-7751-140336910927312\" (UID: \"8b7f830d-d111-4967-a555-864bad5bf852\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.359154 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.359089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87vpr\" (UniqueName: \"kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr\") pod \"test-curl-7751-140336910927312\" (UID: \"8b7f830d-d111-4967-a555-864bad5bf852\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.368595 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.368576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87vpr\" (UniqueName: \"kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr\") pod \"test-curl-7751-140336910927312\" (UID: \"8b7f830d-d111-4967-a555-864bad5bf852\") " pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.387735 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.387715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.527018 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.526991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:06.529138 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:47:06.529103    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b7f830d_d111_4967_a555_864bad5bf852.slice/crio-4e2969f33423b0aa9a8283ef298a94d1fb881fd565822804d9e675a48c583cc3 WatchSource:0}: Error finding container 4e2969f33423b0aa9a8283ef298a94d1fb881fd565822804d9e675a48c583cc3: Status 404 returned error can't find the container with id 4e2969f33423b0aa9a8283ef298a94d1fb881fd565822804d9e675a48c583cc3
Jul 23 19:47:06.746506 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.746426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv" event={"ID":"89ebf26a-5720-4e9b-b134-88d59998d27b","Type":"ContainerDied","Data":"d89bb5d11603001305fa7bca8c9e26efb41a3d5fa9be481c4ecdc242fdad4034"}
Jul 23 19:47:06.746506 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.746436    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"
Jul 23 19:47:06.746506 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.746477    2563 scope.go:117] "RemoveContainer" containerID="4534fcf76d41a56f8f050adfd7fd6793db61a4a8d37afdaf4a0265098ca7e40d"
Jul 23 19:47:06.747955 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.747929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910927312" event={"ID":"8b7f830d-d111-4967-a555-864bad5bf852","Type":"ContainerStarted","Data":"4e2969f33423b0aa9a8283ef298a94d1fb881fd565822804d9e675a48c583cc3"}
Jul 23 19:47:06.749318 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.749301    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:06.749428 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.749320    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7314a02f40fadccc22f4cb9cd75d5b86f1e5f7df6ace8e818bf81fa21f028894"
Jul 23 19:47:06.775223 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.775201    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:47:06.780000 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.779980    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4a426a/rate-test-model-4a426a-kserve-667df94c4c-zz4hv"]
Jul 23 19:47:06.781240 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:06.781197    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7751-140336910927312" oldPodUID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" podUID="8b7f830d-d111-4967-a555-864bad5bf852"
Jul 23 19:47:07.521950 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:07.521920    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ccaf81-2afb-4f4d-a184-a8433c4493ce" path="/var/lib/kubelet/pods/61ccaf81-2afb-4f4d-a184-a8433c4493ce/volumes"
Jul 23 19:47:07.522335 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:07.522318    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74859b76-ebed-4481-81c1-0bc59faa6da9" path="/var/lib/kubelet/pods/74859b76-ebed-4481-81c1-0bc59faa6da9/volumes"
Jul 23 19:47:07.522701 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:07.522688    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ebf26a-5720-4e9b-b134-88d59998d27b" path="/var/lib/kubelet/pods/89ebf26a-5720-4e9b-b134-88d59998d27b/volumes"
Jul 23 19:47:07.756706 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:07.756674    2563 generic.go:356] "Generic (PLEG): container finished" podID="8b7f830d-d111-4967-a555-864bad5bf852" containerID="d266cd5255b60711c99c4792fa9dab812f49e6ad05af231ef7a403ea3bed851d" exitCode=0
Jul 23 19:47:07.757147 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:07.756766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7751-140336910927312" event={"ID":"8b7f830d-d111-4967-a555-864bad5bf852","Type":"ContainerDied","Data":"d266cd5255b60711c99c4792fa9dab812f49e6ad05af231ef7a403ea3bed851d"}
Jul 23 19:47:08.901721 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:08.901696    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:08.934169 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:08.934150    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7751-140336910927312_8b7f830d-d111-4967-a555-864bad5bf852/test-curl-7751-140336910927312/0.log"
Jul 23 19:47:08.965683 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:08.965659    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:08.971245 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:08.971224    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7751-140336910927312"]
Jul 23 19:47:09.079079 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.079015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87vpr\" (UniqueName: \"kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr\") pod \"8b7f830d-d111-4967-a555-864bad5bf852\" (UID: \"8b7f830d-d111-4967-a555-864bad5bf852\") "
Jul 23 19:47:09.081058 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.081037    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr" (OuterVolumeSpecName: "kube-api-access-87vpr") pod "8b7f830d-d111-4967-a555-864bad5bf852" (UID: "8b7f830d-d111-4967-a555-864bad5bf852"). InnerVolumeSpecName "kube-api-access-87vpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:09.180596 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.180568    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87vpr\" (UniqueName: \"kubernetes.io/projected/8b7f830d-d111-4967-a555-864bad5bf852-kube-api-access-87vpr\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:09.197265 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.197240    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"]
Jul 23 19:47:09.197608 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.197596    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7f830d-d111-4967-a555-864bad5bf852" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:09.197646 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.197610    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7f830d-d111-4967-a555-864bad5bf852" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:09.197686 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.197667    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7f830d-d111-4967-a555-864bad5bf852" containerName="test-curl-7751-140336910927312"
Jul 23 19:47:09.200564 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.200550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:09.205467 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.205446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:47:09.212918 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.212890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"]
Jul 23 19:47:09.382421 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.382350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm\" (UID: \"8378c53b-1c55-48ec-a122-d21caf1014cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:09.483641 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.483582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm\" (UID: \"8378c53b-1c55-48ec-a122-d21caf1014cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:09.484225 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.484207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm\" (UID: \"8378c53b-1c55-48ec-a122-d21caf1014cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:09.509504 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.509487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:09.523459 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.523437    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7f830d-d111-4967-a555-864bad5bf852" path="/var/lib/kubelet/pods/8b7f830d-d111-4967-a555-864bad5bf852/volumes"
Jul 23 19:47:09.643401 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.643379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"]
Jul 23 19:47:09.644949 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:47:09.644920    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8378c53b_1c55_48ec_a122_d21caf1014cb.slice/crio-3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088 WatchSource:0}: Error finding container 3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088: Status 404 returned error can't find the container with id 3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088
Jul 23 19:47:09.769393 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.769362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm" event={"ID":"8378c53b-1c55-48ec-a122-d21caf1014cb","Type":"ContainerStarted","Data":"3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088"}
Jul 23 19:47:09.770572 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.770548    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7751-140336910927312"
Jul 23 19:47:09.770667 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:09.770573    2563 scope.go:117] "RemoveContainer" containerID="d266cd5255b60711c99c4792fa9dab812f49e6ad05af231ef7a403ea3bed851d"
Jul 23 19:47:10.778294 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:10.778265    2563 generic.go:356] "Generic (PLEG): container finished" podID="8378c53b-1c55-48ec-a122-d21caf1014cb" containerID="3be1daa42dfc26fac0a9ae0eb38acb393375edd0aeec5b818eed1c1bac98cdbb" exitCode=0
Jul 23 19:47:10.778651 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:10.778329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm" event={"ID":"8378c53b-1c55-48ec-a122-d21caf1014cb","Type":"ContainerDied","Data":"3be1daa42dfc26fac0a9ae0eb38acb393375edd0aeec5b818eed1c1bac98cdbb"}
Jul 23 19:47:11.919953 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:11.919930    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:12.107747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.107678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca\") pod \"8378c53b-1c55-48ec-a122-d21caf1014cb\" (UID: \"8378c53b-1c55-48ec-a122-d21caf1014cb\") "
Jul 23 19:47:12.108035 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.108013    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8378c53b-1c55-48ec-a122-d21caf1014cb" (UID: "8378c53b-1c55-48ec-a122-d21caf1014cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:12.208307 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.208282    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8378c53b-1c55-48ec-a122-d21caf1014cb-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:12.791935 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.791901    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"
Jul 23 19:47:12.792103 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.791900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm" event={"ID":"8378c53b-1c55-48ec-a122-d21caf1014cb","Type":"ContainerDied","Data":"3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088"}
Jul 23 19:47:12.792103 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:12.792030    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c777ec39952c6b32ea90f0ebb601c5fcf872d10eec8c30b955448deca2ac088"
Jul 23 19:47:19.599966 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.599933    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:47:19.600423 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.600141    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" podUID="014fd71e-17be-403f-8736-b2027a46435c" containerName="maas-api" containerID="cri-o://6648590038e7d2ceb440c6c51d6eef42f0c50515d6134eeb4c4e495df0352b65" gracePeriod=30
Jul 23 19:47:19.830681 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.830646    2563 generic.go:356] "Generic (PLEG): container finished" podID="014fd71e-17be-403f-8736-b2027a46435c" containerID="6648590038e7d2ceb440c6c51d6eef42f0c50515d6134eeb4c4e495df0352b65" exitCode=0
Jul 23 19:47:19.830824 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.830717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" event={"ID":"014fd71e-17be-403f-8736-b2027a46435c","Type":"ContainerDied","Data":"6648590038e7d2ceb440c6c51d6eef42f0c50515d6134eeb4c4e495df0352b65"}
Jul 23 19:47:19.859171 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.859118    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:47:19.957234 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.957207    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") pod \"014fd71e-17be-403f-8736-b2027a46435c\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") "
Jul 23 19:47:19.957377 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.957257    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca\") pod \"014fd71e-17be-403f-8736-b2027a46435c\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") "
Jul 23 19:47:19.957377 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.957288    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8d6h\" (UniqueName: \"kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h\") pod \"014fd71e-17be-403f-8736-b2027a46435c\" (UID: \"014fd71e-17be-403f-8736-b2027a46435c\") "
Jul 23 19:47:19.957575 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.957550    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "014fd71e-17be-403f-8736-b2027a46435c" (UID: "014fd71e-17be-403f-8736-b2027a46435c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:19.959181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.959156    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "014fd71e-17be-403f-8736-b2027a46435c" (UID: "014fd71e-17be-403f-8736-b2027a46435c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:47:19.959304 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:19.959284    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h" (OuterVolumeSpecName: "kube-api-access-c8d6h") pod "014fd71e-17be-403f-8736-b2027a46435c" (UID: "014fd71e-17be-403f-8736-b2027a46435c"). InnerVolumeSpecName "kube-api-access-c8d6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:20.058117 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.058054    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014fd71e-17be-403f-8736-b2027a46435c-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:20.058117 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.058077    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014fd71e-17be-403f-8736-b2027a46435c-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:20.058117 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.058088    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8d6h\" (UniqueName: \"kubernetes.io/projected/014fd71e-17be-403f-8736-b2027a46435c-kube-api-access-c8d6h\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:20.837775 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.837742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x" event={"ID":"014fd71e-17be-403f-8736-b2027a46435c","Type":"ContainerDied","Data":"2bc6a0308da4cd72c3efcf1cf7fdcbe7224226377cd175100791df256a8089aa"}
Jul 23 19:47:20.838206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.837804    2563 scope.go:117] "RemoveContainer" containerID="6648590038e7d2ceb440c6c51d6eef42f0c50515d6134eeb4c4e495df0352b65"
Jul 23 19:47:20.838206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.837758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"
Jul 23 19:47:20.859092 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.859069    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:47:20.862980 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:20.862957    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d5ee7b-6ddb6d758f-cdl9x"]
Jul 23 19:47:21.522126 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:21.522094    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="014fd71e-17be-403f-8736-b2027a46435c" path="/var/lib/kubelet/pods/014fd71e-17be-403f-8736-b2027a46435c/volumes"
Jul 23 19:47:22.829171 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:22.829138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:47:24.260692 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:24.260635    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"]
Jul 23 19:47:24.263280 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:24.263260    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d5ee7b-vngcm"]
Jul 23 19:47:25.521685 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:25.521653    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8378c53b-1c55-48ec-a122-d21caf1014cb" path="/var/lib/kubelet/pods/8378c53b-1c55-48ec-a122-d21caf1014cb/volumes"
Jul 23 19:47:31.551795 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.551759    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"]
Jul 23 19:47:31.552288 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552267    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="014fd71e-17be-403f-8736-b2027a46435c" containerName="maas-api"
Jul 23 19:47:31.552373 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552292    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="014fd71e-17be-403f-8736-b2027a46435c" containerName="maas-api"
Jul 23 19:47:31.552373 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552330    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8378c53b-1c55-48ec-a122-d21caf1014cb" containerName="revoke-keys"
Jul 23 19:47:31.552373 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552341    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8378c53b-1c55-48ec-a122-d21caf1014cb" containerName="revoke-keys"
Jul 23 19:47:31.552523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552439    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8378c53b-1c55-48ec-a122-d21caf1014cb" containerName="revoke-keys"
Jul 23 19:47:31.552523 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.552454    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="014fd71e-17be-403f-8736-b2027a46435c" containerName="maas-api"
Jul 23 19:47:31.557573 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.557552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:31.560144 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.560123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\""
Jul 23 19:47:31.562562 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.562536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"]
Jul 23 19:47:31.634569 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.634540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg\" (UID: \"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:31.735439 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.735410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg\" (UID: \"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:31.736096 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.736075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg\" (UID: \"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:31.869186 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.869124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:31.997750 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:31.997722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"]
Jul 23 19:47:31.999475 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:47:31.999450    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8ad1e1_07c0_4b41_9aee_539f4b195d7e.slice/crio-556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10 WatchSource:0}: Error finding container 556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10: Status 404 returned error can't find the container with id 556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10
Jul 23 19:47:32.902139 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:32.902106    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" containerID="219c2325663fe4c6e307c5f928a9979ea17e06ee1fa1470610bd232c1d139cda" exitCode=0
Jul 23 19:47:32.902518 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:32.902190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg" event={"ID":"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e","Type":"ContainerDied","Data":"219c2325663fe4c6e307c5f928a9979ea17e06ee1fa1470610bd232c1d139cda"}
Jul 23 19:47:32.902518 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:32.902242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg" event={"ID":"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e","Type":"ContainerStarted","Data":"556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10"}
Jul 23 19:47:34.041078 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.041055    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:34.050796 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.050777    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca\") pod \"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e\" (UID: \"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e\") "
Jul 23 19:47:34.051123 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.051104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" (UID: "ac8ad1e1-07c0-4b41-9aee-539f4b195d7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:34.151488 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.151460    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e-maas-api-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:34.914737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.914703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg" event={"ID":"ac8ad1e1-07c0-4b41-9aee-539f4b195d7e","Type":"ContainerDied","Data":"556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10"}
Jul 23 19:47:34.914737 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.914723    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"
Jul 23 19:47:34.914978 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:34.914748    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="556934769ef707a81c799afc30b78dc7b9e369cf446a7572e5b02fb97d182b10"
Jul 23 19:47:41.918282 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:41.918246    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:47:41.920960 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:41.918482    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" podUID="87dff83d-92cd-440b-8f31-50ebf01386ac" containerName="maas-api" containerID="cri-o://d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae" gracePeriod=30
Jul 23 19:47:42.165782 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.165752    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:47:42.207643 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.207551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j75sw\" (UniqueName: \"kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw\") pod \"87dff83d-92cd-440b-8f31-50ebf01386ac\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") "
Jul 23 19:47:42.207774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.207688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca\") pod \"87dff83d-92cd-440b-8f31-50ebf01386ac\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") "
Jul 23 19:47:42.207774 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.207717    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls\") pod \"87dff83d-92cd-440b-8f31-50ebf01386ac\" (UID: \"87dff83d-92cd-440b-8f31-50ebf01386ac\") "
Jul 23 19:47:42.208038 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.208011    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87dff83d-92cd-440b-8f31-50ebf01386ac" (UID: "87dff83d-92cd-440b-8f31-50ebf01386ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:42.209718 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.209694    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87dff83d-92cd-440b-8f31-50ebf01386ac" (UID: "87dff83d-92cd-440b-8f31-50ebf01386ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:47:42.209988 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.209971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw" (OuterVolumeSpecName: "kube-api-access-j75sw") pod "87dff83d-92cd-440b-8f31-50ebf01386ac" (UID: "87dff83d-92cd-440b-8f31-50ebf01386ac"). InnerVolumeSpecName "kube-api-access-j75sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:42.308512 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.308480    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87dff83d-92cd-440b-8f31-50ebf01386ac-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:42.308512 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.308512    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87dff83d-92cd-440b-8f31-50ebf01386ac-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:42.308512 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.308522    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j75sw\" (UniqueName: \"kubernetes.io/projected/87dff83d-92cd-440b-8f31-50ebf01386ac-kube-api-access-j75sw\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\""
Jul 23 19:47:42.962020 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.961985    2563 generic.go:356] "Generic (PLEG): container finished" podID="87dff83d-92cd-440b-8f31-50ebf01386ac" containerID="d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae" exitCode=0
Jul 23 19:47:42.962432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.962034    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"
Jul 23 19:47:42.962432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.962075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" event={"ID":"87dff83d-92cd-440b-8f31-50ebf01386ac","Type":"ContainerDied","Data":"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"}
Jul 23 19:47:42.962432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.962127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt" event={"ID":"87dff83d-92cd-440b-8f31-50ebf01386ac","Type":"ContainerDied","Data":"0b29891923c371d1aa3060dbdc0e57805a3bbb2c40cc804f715ff4a1f6145d75"}
Jul 23 19:47:42.962432 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.962149    2563 scope.go:117] "RemoveContainer" containerID="d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"
Jul 23 19:47:42.971713 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.971697    2563 scope.go:117] "RemoveContainer" containerID="d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"
Jul 23 19:47:42.971990 ip-10-0-135-11 kubenswrapper[2563]: E0723 19:47:42.971972    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae\": container with ID starting with d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae not found: ID does not exist" containerID="d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"
Jul 23 19:47:42.972046 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.971998    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae"} err="failed to get container status \"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae\": rpc error: code = NotFound desc = could not find container \"d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae\": container with ID starting with d08a2379515c68f576689a978b6b962cd41cb594860059a1e34534a6b8c74eae not found: ID does not exist"
Jul 23 19:47:42.982924 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.982904    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:47:42.986822 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:42.986794    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4a426a-7659d49f94-kmfxt"]
Jul 23 19:47:43.522259 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:43.522225    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87dff83d-92cd-440b-8f31-50ebf01386ac" path="/var/lib/kubelet/pods/87dff83d-92cd-440b-8f31-50ebf01386ac/volumes"
Jul 23 19:47:45.019238 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:45.019209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5xfcx"]
Jul 23 19:47:46.683118 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:46.683087    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"]
Jul 23 19:47:46.686844 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:46.686823    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4a426a-r56qg"]
Jul 23 19:47:47.522370 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:47.522337    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" path="/var/lib/kubelet/pods/ac8ad1e1-07c0-4b41-9aee-539f4b195d7e/volumes"
Jul 23 19:47:57.952535 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:57.952502    2563 scope.go:117] "RemoveContainer" containerID="c78731e3a3848deebd243727e61dbe98159b689a2e866ff279277a87965de495"
Jul 23 19:47:59.168291 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:59.168263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-75zkd_dd84eb65-3b47-4676-bcbe-05c787b12797/manager/0.log"
Jul 23 19:47:59.282987 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:59.282964    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nvx42_55e15c55-30aa-4801-a6c0-d3af4661000c/manager/0.log"
Jul 23 19:47:59.523850 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:59.523822    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7z4q8_3c65828c-5904-4308-a58f-92078ea2d3d7/server/0.log"
Jul 23 19:47:59.642224 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:47:59.642200    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-z8ql5_38a7bfef-a3e8-485f-a008-14fcbe7d4988/manager/0.log"
Jul 23 19:48:00.021235 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:00.021213    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-qhjvh_7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0/copy-manifests/0.log"
Jul 23 19:48:00.027257 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:00.027239    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-qhjvh_7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0/manager/0.log"
Jul 23 19:48:00.705883 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:00.705842    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747265-kwb2q_2af0e9bc-5ab3-4a39-b08f-612f00aaaf59/cleanup/0.log"
Jul 23 19:48:01.554181 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.554156    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/util/0.log"
Jul 23 19:48:01.560430 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.560411    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/pull/0.log"
Jul 23 19:48:01.565862 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.565842    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/extract/0.log"
Jul 23 19:48:01.673118 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.673091    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/util/0.log"
Jul 23 19:48:01.678697 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.678681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/pull/0.log"
Jul 23 19:48:01.684435 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.684416    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/extract/0.log"
Jul 23 19:48:01.789796 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.789774    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/util/0.log"
Jul 23 19:48:01.795282 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.795263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/pull/0.log"
Jul 23 19:48:01.800653 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.800639    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/extract/0.log"
Jul 23 19:48:01.902957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.902904    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/util/0.log"
Jul 23 19:48:01.909542 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.909521    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/pull/0.log"
Jul 23 19:48:01.914747 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:01.914731    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/extract/0.log"
Jul 23 19:48:02.144827 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.144778    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q8bw5_8d8ea57f-ece5-4df3-9974-c7dcf851d649/manager/0.log"
Jul 23 19:48:02.255226 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.255204    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t28bf_22fcaa95-ed7f-47ba-9392-385a8c457eec/manager/0.log"
Jul 23 19:48:02.361158 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.361135    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6x2tp_9ea53e04-ba51-48e2-99de-7ee1fdb87fd4/kuadrant-console-plugin/0.log"
Jul 23 19:48:02.475148 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.475120    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bdlbn_41079020-1b5d-4d01-bf75-dede8f45852e/registry-server/0.log"
Jul 23 19:48:02.699645 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.699555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5xfcx_e0c60f38-b099-4d14-be18-0ab0a9404e3c/limitador/0.log"
Jul 23 19:48:02.818136 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:02.818103    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xnb74_30c219e9-d394-4f87-b2a5-2e8cb5ffd79c/manager/0.log"
Jul 23 19:48:03.368368 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:03.368341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74b646958c-5gfz4_8b628d55-fa54-405a-a062-2c2498d640d6/kube-auth-proxy/0.log"
Jul 23 19:48:04.371329 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:04.371304    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vh2kx_b2581274-426e-4405-ba6d-da9973234bea/main/0.log"
Jul 23 19:48:04.708969 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:04.708895    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-v9q7w_49b23f24-94ee-4006-b191-13e56585b64a/main/0.log"
Jul 23 19:48:04.814082 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:04.814055    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2p7j9_0f477e24-d34e-41ab-900b-84984d94ded4/main/0.log"
Jul 23 19:48:10.209175 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209141    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ctxw/must-gather-vfh24"]
Jul 23 19:48:10.209559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209466    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" containerName="revoke-keys"
Jul 23 19:48:10.209559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209476    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" containerName="revoke-keys"
Jul 23 19:48:10.209559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209489    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87dff83d-92cd-440b-8f31-50ebf01386ac" containerName="maas-api"
Jul 23 19:48:10.209559 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209539    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="87dff83d-92cd-440b-8f31-50ebf01386ac" containerName="maas-api"
Jul 23 19:48:10.209726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209618    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="87dff83d-92cd-440b-8f31-50ebf01386ac" containerName="maas-api"
Jul 23 19:48:10.209726 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.209632    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac8ad1e1-07c0-4b41-9aee-539f4b195d7e" containerName="revoke-keys"
Jul 23 19:48:10.212931 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.212911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.215153 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.215134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ctxw\"/\"kube-root-ca.crt\""
Jul 23 19:48:10.215748 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.215729    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ctxw\"/\"default-dockercfg-vxjpx\""
Jul 23 19:48:10.215805 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.215746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ctxw\"/\"openshift-service-ca.crt\""
Jul 23 19:48:10.222531 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.222509    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/must-gather-vfh24"]
Jul 23 19:48:10.327490 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.327468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6818320e-2078-4686-b4cd-9b3215eeaaa8-must-gather-output\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.327651 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.327505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57qg\" (UniqueName: \"kubernetes.io/projected/6818320e-2078-4686-b4cd-9b3215eeaaa8-kube-api-access-z57qg\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.428080 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.428054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6818320e-2078-4686-b4cd-9b3215eeaaa8-must-gather-output\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.428206 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.428086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57qg\" (UniqueName: \"kubernetes.io/projected/6818320e-2078-4686-b4cd-9b3215eeaaa8-kube-api-access-z57qg\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.428411 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.428391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6818320e-2078-4686-b4cd-9b3215eeaaa8-must-gather-output\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.436282 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.436252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57qg\" (UniqueName: \"kubernetes.io/projected/6818320e-2078-4686-b4cd-9b3215eeaaa8-kube-api-access-z57qg\") pod \"must-gather-vfh24\" (UID: \"6818320e-2078-4686-b4cd-9b3215eeaaa8\") " pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.522265 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.522242    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/must-gather-vfh24"
Jul 23 19:48:10.653619 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:10.653598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/must-gather-vfh24"]
Jul 23 19:48:10.655664 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:48:10.655612    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6818320e_2078_4686_b4cd_9b3215eeaaa8.slice/crio-2298d3595b62a6c9f0f3fd64a15f4d2b02983ebd5be9f175dedd1ae680600a72 WatchSource:0}: Error finding container 2298d3595b62a6c9f0f3fd64a15f4d2b02983ebd5be9f175dedd1ae680600a72: Status 404 returned error can't find the container with id 2298d3595b62a6c9f0f3fd64a15f4d2b02983ebd5be9f175dedd1ae680600a72
Jul 23 19:48:11.109711 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:11.109676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/must-gather-vfh24" event={"ID":"6818320e-2078-4686-b4cd-9b3215eeaaa8","Type":"ContainerStarted","Data":"2298d3595b62a6c9f0f3fd64a15f4d2b02983ebd5be9f175dedd1ae680600a72"}
Jul 23 19:48:12.120353 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:12.120312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/must-gather-vfh24" event={"ID":"6818320e-2078-4686-b4cd-9b3215eeaaa8","Type":"ContainerStarted","Data":"e4b13fc57ed216cfcd7a6b0aad6792dd3c7133e56042e0860c2bcf9d50f52101"}
Jul 23 19:48:12.120832 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:12.120368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/must-gather-vfh24" event={"ID":"6818320e-2078-4686-b4cd-9b3215eeaaa8","Type":"ContainerStarted","Data":"b3c37b71e0b828abefa605f8781f5d6f306ad5237dd5898e2f12cd3b3d094a17"}
Jul 23 19:48:12.137972 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:12.137916    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ctxw/must-gather-vfh24" podStartSLOduration=1.059637266 podStartE2EDuration="2.137895477s" totalImagesPullingTime="1.078258211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:48:12.136018955 +0000 UTC m=+2835.038605060" watchObservedRunningTime="2026-07-23 19:48:12.137895477 +0000 UTC m=+2835.040481555"
Jul 23 19:48:13.328785 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:13.328757    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fwvxh_4a3e0324-af23-4a58-9836-35b49754f300/global-pull-secret-syncer/0.log"
Jul 23 19:48:13.434289 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:13.434258    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wf4ks_9e0b68d4-6d40-4b89-9e98-34232914551f/konnectivity-agent/0.log"
Jul 23 19:48:13.480741 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:13.480704    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-11.ec2.internal_0cf1e41fbd8c7e6ff0f79ac186a0f32c/haproxy/0.log"
Jul 23 19:48:18.006631 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.006595    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/extract/0.log"
Jul 23 19:48:18.030990 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.030805    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/util/0.log"
Jul 23 19:48:18.056856 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.056789    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fcg9h_f94d4809-4657-4f15-9f61-423dfaa0b77f/pull/0.log"
Jul 23 19:48:18.102759 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.102731    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/extract/0.log"
Jul 23 19:48:18.127242 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.127214    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/util/0.log"
Jul 23 19:48:18.159171 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.159145    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fq6gv_a62aa6d5-b444-4f57-8208-179aa960583e/pull/0.log"
Jul 23 19:48:18.195278 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.195253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/extract/0.log"
Jul 23 19:48:18.216884 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.216859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/util/0.log"
Jul 23 19:48:18.240324 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.240295    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l4947_7c51f2dd-f7c7-40fb-a48d-2cf89d49ee9b/pull/0.log"
Jul 23 19:48:18.267639 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.267521    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/extract/0.log"
Jul 23 19:48:18.294964 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.294930    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/util/0.log"
Jul 23 19:48:18.320149 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.320115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18mphg_97451924-94b7-41ed-960e-d02e1054172c/pull/0.log"
Jul 23 19:48:18.685751 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.685674    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q8bw5_8d8ea57f-ece5-4df3-9974-c7dcf851d649/manager/0.log"
Jul 23 19:48:18.715305 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.715271    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t28bf_22fcaa95-ed7f-47ba-9392-385a8c457eec/manager/0.log"
Jul 23 19:48:18.739777 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.739753    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6x2tp_9ea53e04-ba51-48e2-99de-7ee1fdb87fd4/kuadrant-console-plugin/0.log"
Jul 23 19:48:18.791530 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.791501    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bdlbn_41079020-1b5d-4d01-bf75-dede8f45852e/registry-server/0.log"
Jul 23 19:48:18.941856 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:18.941761    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5xfcx_e0c60f38-b099-4d14-be18-0ab0a9404e3c/limitador/0.log"
Jul 23 19:48:19.039281 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:19.039246    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xnb74_30c219e9-d394-4f87-b2a5-2e8cb5ffd79c/manager/0.log"
Jul 23 19:48:21.129394 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:21.129359    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh7zb_4d2540a2-73c6-4f3f-895b-62856e50d2b5/node-exporter/0.log"
Jul 23 19:48:21.151886 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:21.151781    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh7zb_4d2540a2-73c6-4f3f-895b-62856e50d2b5/kube-rbac-proxy/0.log"
Jul 23 19:48:21.173099 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:21.173076    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh7zb_4d2540a2-73c6-4f3f-895b-62856e50d2b5/init-textfile/0.log"
Jul 23 19:48:22.858846 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:22.858801    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-kwtnv_bf4fa8f2-012d-4e36-9925-2f09feba437a/networking-console-plugin/0.log"
Jul 23 19:48:23.372616 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.372572    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"]
Jul 23 19:48:23.381209 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.381176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.387957 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.387925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"]
Jul 23 19:48:23.465749 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.465708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-proc\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.465951 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.465760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-lib-modules\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.465951 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.465792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-sys\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.465951 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.465849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4cb5\" (UniqueName: \"kubernetes.io/projected/b44339e3-c073-444b-afff-5f1389288ad6-kube-api-access-p4cb5\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.466140 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.465970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-podres\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567191 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-podres\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-proc\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-proc\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567371 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-podres\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-lib-modules\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-sys\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4cb5\" (UniqueName: \"kubernetes.io/projected/b44339e3-c073-444b-afff-5f1389288ad6-kube-api-access-p4cb5\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567545 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-lib-modules\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.567788 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.567619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44339e3-c073-444b-afff-5f1389288ad6-sys\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.575764 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.575730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4cb5\" (UniqueName: \"kubernetes.io/projected/b44339e3-c073-444b-afff-5f1389288ad6-kube-api-access-p4cb5\") pod \"perf-node-gather-daemonset-rflfc\" (UID: \"b44339e3-c073-444b-afff-5f1389288ad6\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.695418 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.695330    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:23.894343 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:23.894315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"]
Jul 23 19:48:23.897111 ip-10-0-135-11 kubenswrapper[2563]: W0723 19:48:23.897077    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb44339e3_c073_444b_afff_5f1389288ad6.slice/crio-db17ef1523f6416d847fc8c64d75a1b2ffb4fb25750d939ba2f04d1b6e78f661 WatchSource:0}: Error finding container db17ef1523f6416d847fc8c64d75a1b2ffb4fb25750d939ba2f04d1b6e78f661: Status 404 returned error can't find the container with id db17ef1523f6416d847fc8c64d75a1b2ffb4fb25750d939ba2f04d1b6e78f661
Jul 23 19:48:24.174701 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:24.174673    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-2wh8f_3a96e80c-5555-42f4-9d10-dfb55851a3bb/download-server/0.log"
Jul 23 19:48:24.224667 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:24.224625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc" event={"ID":"b44339e3-c073-444b-afff-5f1389288ad6","Type":"ContainerStarted","Data":"96840dd50eb20fc229f9cdaf97f38c456e4f6c9d364c6e2ab1bbb5a77506a67f"}
Jul 23 19:48:24.224910 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:24.224690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc" event={"ID":"b44339e3-c073-444b-afff-5f1389288ad6","Type":"ContainerStarted","Data":"db17ef1523f6416d847fc8c64d75a1b2ffb4fb25750d939ba2f04d1b6e78f661"}
Jul 23 19:48:24.224910 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:24.224842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:24.241369 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:24.241316    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc" podStartSLOduration=1.241295511 podStartE2EDuration="1.241295511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:48:24.239999985 +0000 UTC m=+2847.142586078" watchObservedRunningTime="2026-07-23 19:48:24.241295511 +0000 UTC m=+2847.143881589"
Jul 23 19:48:25.854687 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:25.854657    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xg9q2_ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740/dns/0.log"
Jul 23 19:48:25.875463 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:25.875427    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xg9q2_ee6a0efc-a9f2-481d-89c9-cfe2dd8cd740/kube-rbac-proxy/0.log"
Jul 23 19:48:25.952244 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:25.952215    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rcdmw_8bda1177-dc5b-41fd-849d-f590926553b7/dns-node-resolver/0.log"
Jul 23 19:48:26.474019 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:26.473985    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7dfd8c47f4-6mgfg_29e33cca-e56c-47d5-8796-81f26f1ab1a2/registry/0.log"
Jul 23 19:48:26.545407 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:26.545375    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gm8mt_9bbb4dde-6efd-4b8c-98c1-d2ec0124d5f7/node-ca/0.log"
Jul 23 19:48:27.652972 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:27.652947    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74b646958c-5gfz4_8b628d55-fa54-405a-a062-2c2498d640d6/kube-auth-proxy/0.log"
Jul 23 19:48:28.539661 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:28.539609    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dxxx6_44d7b1f5-0ee1-48af-8ea1-17059eb3ba0c/serve-healthcheck-canary/0.log"
Jul 23 19:48:29.087103 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:29.087072    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqzqj_77b04f81-40f3-4a86-a02f-1fc31f770d1e/kube-rbac-proxy/0.log"
Jul 23 19:48:29.107863 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:29.107836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqzqj_77b04f81-40f3-4a86-a02f-1fc31f770d1e/exporter/0.log"
Jul 23 19:48:29.133299 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:29.133271    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqzqj_77b04f81-40f3-4a86-a02f-1fc31f770d1e/extractor/0.log"
Jul 23 19:48:30.242757 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:30.242728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-rflfc"
Jul 23 19:48:31.244905 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.244877    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-75zkd_dd84eb65-3b47-4676-bcbe-05c787b12797/manager/0.log"
Jul 23 19:48:31.499517 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.499489    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nvx42_55e15c55-30aa-4801-a6c0-d3af4661000c/manager/0.log"
Jul 23 19:48:31.639963 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.639934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7z4q8_3c65828c-5904-4308-a58f-92078ea2d3d7/server/0.log"
Jul 23 19:48:31.728315 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.728288    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-z8ql5_38a7bfef-a3e8-485f-a008-14fcbe7d4988/manager/0.log"
Jul 23 19:48:31.908321 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.908291    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-qhjvh_7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0/manager/0.log"
Jul 23 19:48:31.929114 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:31.928984    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-qhjvh_7dbda8d0-bd26-45c6-8b86-cc5e4a5068d0/copy-manifests/0.log"
Jul 23 19:48:33.478336 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:33.478254    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74b646958c-5gfz4_8b628d55-fa54-405a-a062-2c2498d640d6/kube-auth-proxy/0.log"
Jul 23 19:48:34.907233 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:34.907205    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74b646958c-5gfz4_8b628d55-fa54-405a-a062-2c2498d640d6/kube-auth-proxy/0.log"
Jul 23 19:48:38.352658 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:38.352631    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-c9d8f_6f9cb974-5a29-4c83-a121-1752f84ee609/migrator/0.log"
Jul 23 19:48:38.374556 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:38.374530    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-c9d8f_6f9cb974-5a29-4c83-a121-1752f84ee609/graceful-termination/0.log"
Jul 23 19:48:39.752836 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.752770    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4x4ss_cbc7db65-d860-49cc-9d8c-8273119557e4/kube-multus/0.log"
Jul 23 19:48:39.834584 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.834555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/kube-multus-additional-cni-plugins/0.log"
Jul 23 19:48:39.855857 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.855806    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/egress-router-binary-copy/0.log"
Jul 23 19:48:39.876977 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.876953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/cni-plugins/0.log"
Jul 23 19:48:39.896066 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.896046    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/bond-cni-plugin/0.log"
Jul 23 19:48:39.916360 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.916343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/routeoverride-cni/0.log"
Jul 23 19:48:39.940950 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.940928    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/whereabouts-cni-bincopy/0.log"
Jul 23 19:48:39.961101 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:39.961085    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbldv_e0f03efa-faba-4a1a-8541-652b8324483f/whereabouts-cni/0.log"
Jul 23 19:48:40.315231 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:40.315203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9s5gk_d05c0c74-6f50-4551-a0c2-6c8da9b2eace/network-metrics-daemon/0.log"
Jul 23 19:48:40.336803 ip-10-0-135-11 kubenswrapper[2563]: I0723 19:48:40.336769    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9s5gk_d05c0c74-6f50-4551-a0c2-6c8da9b2eace/kube-rbac-proxy/0.log"